Systems and methods for adjusting UAV trajectory

ABSTRACT

A system for controlling an unmanned aerial vehicle (UAV) includes a first user interface configured to receive a first user input and a second user interface configured to receive a second user input. The first user input provides one or more instructions to effect an autonomous flight of the UAV. The second user input provides one or more instructions to modify the autonomous flight of the UAV, The autonomous flight includes a flight towards a target.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of International Application No.PCT/CN2016/074686, filed on Feb. 26, 2016, the entire contents of whichare incorporated herein by reference.

BACKGROUND

Aerial vehicles have a wide range of real-world applications includingsurveillance, reconnaissance, exploration, logistics transport, disasterrelief, aerial photography, large-scale agriculture automation, livevideo broadcasting, etc. In some applications, an aerial vehiclecarrying a payload (e.g., a camera) may be controlled to fly around atarget to acquire data or perform certain tasks. With the advancement ofsensors and navigation technologies, autonomous flight or control of theaerial vehicles has become possible. The usefulness of aerial vehicleswith autonomous flight may be improved.

SUMMARY

Presently, aerial vehicles may fly along a preset trajectory orautonomously planned trajectories during autonomous flight. Examples ofautonomous flight may include autonomous return of the aerial vehicles,autonomous navigation of the aerial vehicles along one or morewaypoints, and/or autonomous flight to a point of interest. Userintervention during the autonomous flight maybe limited, or may disruptthe autonomous flight of the aerial vehicle. In some instances however,ability for the user to quickly intervene or supplement autonomousflight of aerial vehicles may be desired. For example, during anautonomous return flight of the aerial vehicle, user's input may behelpful in avoiding an obstacle such as a building (e.g., if an aerialvehicle had no obstacle avoidance sensors). In addition, in somecircumstances, the user may desire the ability to slightly modify aflight of the aerial vehicle while still relying on the autonomousoperation of the aerial vehicle in accomplishing a given task. Forinstance, a user may wish to deviate from a selected target ordestination.

Accordingly, a need exists for the ability to modify autonomous flightof aerial vehicles. Such ability may be provided via flight controlsystems that are intuitive and easy to use, and that allows a human tomodify and/or affect an autonomous flight of an aerial vehicle throughinteraction with a human-system interface. The burden of manuallypiloting the aerial vehicle on the user can be significantly reduced,while still enabling a degree of control or modification by the userwhen desired or advantageous.

Thus, in one aspect, a system for modifying autonomous flight of anunmanned aerial vehicle (UAV) is provided. The system comprises: a firstuser interface configured to receive a first user input, wherein thefirst user input provides one or more instructions to effect anautonomous flight of the UAV; and a second user interface configured toreceive a second user input, wherein the second user input provides oneor more instructions to modify the autonomous flight of the UAV.

In another aspect, a method of modifying autonomous flight of anunmanned aerial vehicle (UAV) is provided. The method comprises:receiving a first user input at a first user interface, wherein thefirst user input provides one or more instructions to effect anautonomous flight of the UAV; and receiving a second user input at asecond user interface, wherein the second user input provides one ormore instructions to modify the autonomous flight of the UAV.

In another aspect, a non-transitory computer readable medium formodifying flight of an unmanned aerial vehicle (UAV) is provided. Thenon-transitory computer readable medium comprises code, logic, orinstructions to: receive a first user input at a first user interface,wherein the first user input provides one or more instructions to effectan autonomous flight of the UAV; and receive a second user input at asecond user interface, wherein the second user input provides one ormore instructions to modify the autonomous flight of the UAV.

In another aspect, a system for modifying autonomous flight of anunmanned aerial vehicle (UAV) is provided. The system comprises: aflight controller configured to (1) generate a first set of signals thateffect autonomous flight of the UAV in response to a first user inputreceived at a first user interface, and (2) generate a second set ofsignals that modify the autonomous flight of the UAV in response to asecond user input received at a second user interface.

In another aspect, a method of modifying autonomous flight of anunmanned aerial vehicle (UAV) is provided. The method comprises:generating a first set of signals, with aid of a flight controller, thateffect autonomous flight of the UAV in response to a first user inputreceived at a first user interface; and generating a second set ofsignals, with aid of the flight controller, that modify the autonomousflight of the UAV in response to a second user input received at asecond user interface.

In another aspect, a non-transitory computer readable medium formodifying flight of an unmanned aerial vehicle (UAV) is provided. Thenon-transitory computer readable medium comprises code, logic, orinstructions to: generate a first set of signals, with aid of a flightcontroller, that effect autonomous flight of the UAV in response to afirst user input received at a first user interface; and generate asecond set of signals, with aid of the flight stroller, that modify theautonomous flight of the UAV in response to a second user input receivedat a second user interface.

In another aspect, an unmanned aerial vehicle (UAV) is provided. The UAVcomprises: a flight controller configured to generate (1) a first set ofsignals that instruct autonomous flight of the UAV, wherein the firstset of signals are generated based on a first user input received at afirst user interface, and (2) a second set of signals that instructmodification of the autonomous flight of the UAV, wherein the second setof signals are generated based on a second user input received at asecond user interface; and one or more propulsion units configured to(a) effect the autonomous flight of the UAV in response to the first setof signals, and (b) modify the autonomous flight of the UAV in responseto the second set of signals.

In another aspect, a system for modifying flight of an unmanned aerialvehicle (UAV) is provided. The system comprises: one or more processors,individual or collectively configured to: effect an autonomous flight ofthe UAV, wherein the autonomous flight comprises an autonomous flightpath; and modify the autonomous flight path in response to a user input,wherein the autonomous flight path is modified while maintaining theautonomous flight.

In another aspect, a method for modifying flight of an unmanned aerialvehicle (UAV) is provided. The method comprises: effecting an autonomousflight of the UAV, wherein the autonomous flight comprises an autonomousflight path; and modifying the autonomous flight path in response to auser input, wherein the autonomous flight path is modified whilemaintaining the autonomous flight.

In another aspect, a non-transitory computer readable medium formodifying flight of an unmanned aerial vehicle (UAV) is provided. Thenon-transitory computer readable medium comprises code, logic, orinstructions to: effect an autonomous flight of the UAV, wherein theautonomous flight comprises an autonomous flight path; and modify theautonomous flight path in response to a user input, wherein theautonomous flight path is modified while maintaining the autonomousflight.

In another aspect, an unmanned aerial vehicle (UAV) is provided. The UAVcomprises: a flight controller configured to generate (1) a first set ofsignals for autonomous flight of the UAV, wherein the autonomous flightcomprises an autonomous flight path, and (2) a second set of signals formodification of the autonomous flight path, wherein the autonomousflight path is modified while maintaining the autonomous flight; and oneor more propulsion units configured to (a) effect the autonomous flightof the UAV in response to the first set of signals, and (b) modify theautonomous flight path of the UAV in response to the second set ofsignals.

It shall be understood that different aspects of the disclosure can beappreciated individually, collectively, or in combination with eachother. Various aspects of the disclosure described herein may be appliedto any of the particular applications set forth below or for any othertypes of movable objects. Any description herein of an aerial vehiclemay apply to and be used for any movable object, such as any vehicle.Additionally, the systems, devices, and methods disclosed herein in thecontext of aerial motion (e.g., flight) may also be applied in thecontext of other types of motion, such as movement on the ground or onwater, underwater motion, or motion in space.

Other objects and features of the present disclosure will becomeapparent by a review of the specification, claims, and appended figures.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present disclosure will be obtained by reference tothe following detailed description that sets forth illustrativeembodiments, in which the principles of the disclosure are utilized, andthe accompanying drawings of which:

FIG. 1 shows an example of a system used for navigation, in accordancewith embodiments.

FIG. 2 illustrates a user terminal, in accordance with embodiments.

FIG. 3 illustrates a first user interface and a second user interfaceworking in concert, in accordance with embodiments.

FIG. 4 illustrates a method for modifying autonomous flight of a UAV, inaccordance with embodiments.

FIG. 5 illustrates an autonomous flight path of the UAV being modifiedby a user input, in accordance with embodiments.

FIG. 6 illustrates a side view of an autonomous flight path of themovable object modified by a user input, in accordance with embodiments.

FIG. 7 illustrates a force of the user input proportionally modifyingthe autonomous flight of the movable object, in accordance withembodiments.

FIG. 8 illustrates behavior of the UAV upon reaching a threshold, inaccordance with embodiments.

FIG. 9 illustrates behavior of the UAV after user input modifying theautonomous flight of the movable object is released, in accordance withembodiments.

FIG. 10 illustrates a new autonomous flight path of a UAV modified by auser input, in accordance with embodiments.

FIG. 11 illustrates an unmanned aerial vehicle (UAV), in accordance withembodiments.

FIG. 12 is a schematic illustration by way of block diagram of a systemfor controlling a movable object, in accordance with embodiments.

FIG. 13 illustrates a UAV flying in a curved trajectory in response toactuation of one or more control sticks, in accordance with embodiments.

FIG. 14 illustrates a top-down view of an unmanned aerial vehicle movingwith increased or decreased velocity along an autonomous flight path, inaccordance with embodiments.

DETAILED DESCRIPTION

Systems, methods, and devices provided herein can be used to give anoperator the ability to affect and/or modify flight of an autonomouslyoperating aerial vehicle. For example, a user input device may beprovided that is intuitive and easy to use. The user input device may beutilized to modify a flight of the aerial vehicle during autonomousoperation of the aerial vehicle. In some instances, the autonomouslyoperating vehicle may have a predetermined goal. For example, theautonomously operating vehicle may have a predetermined task it is setout to accomplish, or a target it is heading towards. Accordingly, theautonomously operating vehicle may continue in accomplishing itspredetermined task, or continue heading towards its target while itsparameters, such as flight path and/or flight direction, is modifiedaccording to the user input. In some instances, a threshold formodification permitted by a user may be provided in order to ensure thatthe aerial vehicle is able to accomplish its task or arrive at thedestination under autonomous control despite the user input.

In some instances, different user interfaces may be provided foraccepting different types of user inputs. The different user interfacesmay include hardware and/or software interfaces. For example, thedifferent user interfaces may include a physical button on a device orinteractive buttons displayed on a screen. In some instances, thedifferent user interfaces may comprise two different user interfaces.For example, a first user interface may be provided that can be used toimprove the ease of autonomous operation of the aerial vehicles. Thefirst user interface may allow control of an aerial vehicle throughinteraction with a graphical human system interface and significantlyreduce burden of manually piloting the aerial vehicle. The first userinterface may be used in providing the aerial vehicle with autonomoustasks to accomplish. In some instances, the autonomous tasks toaccomplish may be designated with simple commands (e.g., touching atarget on a map). A second user interface may be provided that allowssimple and intuitive modification of the autonomous operation (e.g.,autonomous flight) of the aerial vehicle. For example, while the aerialvehicle is autonomously navigating ds its target, user input on thesecond user interface may slightly modify a trajectory of the aerialvehicle. In some instances, the user input provided on the second userinterface may modify parameters of the autonomous flight while theautonomous flight of the aerial vehicle is maintained.

The ability to affect and/or modify flight of an autonomously operatingaerial vehicle may improve maneuverability of the aerial vehicles underautonomous control. The burden of manually piloting the aerial vehicleon a user can be significantly reduced, yet adjustments (e.g.,modifications) on autonomous flight may be allowed such that wheredesired or beneficial, the user can affect, or modify, the autonomousflight. The separate functionalities provided on different userinterfaces may simplify control such that both skilled, and unskilledusers may take advantage of the benefits enabled by the disclosureprovided herein. In addition, the separate functionalities provided ondifferent user interfaces may ensure that quick actions can be taken tomodify autonomous flight of the aerial vehicles in emergencies orunexpected situations at the second user interface) without confusion orerror.

The ability to modify autonomous flight may be particularly useful whenthere are obstacles encountered by the aerial vehicle that is undetected(e.g., through error or if the aerial vehicle lacks obstacle sensors)which is noticed by a user. In such cases, slight modifications may bemade to the UAV flight without disrupting autonomous operation of theaerial vehicle such that a given task may continue to be accomplishedafter the modification. In some instances, the ability to modifyautonomous flight may be particularly useful if the user desires adegree of deviation from a given flight path while maintainingautonomous flight towards a target destination (e.g., the user seessomething interesting). Flight path as used herein may refer to a paththe aerial vehicle takes during flight. In some instances, the flightpath may refer to a trajectory of the aerial vehicle or a flightdirection of the aerial vehicle (e.g., in two-dimensional orthree-dimensional coordinates). In some instances, the flight path mayrefer to a preconfigured flight path (e.g., trajectory) which the aerialvehicle is set to follow. In some instances, the flight path may referto an instantaneous flight direction of the aerial vehicle.

The user input may affect and/or modify a flight path of the aerialvehicle by adding a directional component to the flight path, and/oradding a velocity or acceleration component to the aerial vehicle. Insome instances, the ability to modify autonomous flight may beparticularly useful if the user desires the UAV to fly in certain flightpatterns or to perform maneuverings (e.g., making the UAV fly in anascending or descending spiral) that are not easily implemented. Itshould be noted that the ability to modify autonomous flight can beincorporated into any type of aerial vehicle, as well as any vehiclethat is capable of traversing air, water, land, and/or space.

It shall be understood that different aspects of the disclosure can beappreciated individually, collectively, or in combination with eachother. Various aspects of the disclosure described herein may be appliedto any of the particular applications set forth below or for any othertypes of remotely controlled vehicles or movable objects.

FIG. 1 shows an example of a system used for navigation, in accordancewith embodiments. The navigation system may include a movable object 100and a user terminal 106 capable of communicating with the movableobject. The movable object may be configured to carry a payload 104. Theuser terminal may be used to control one or more motion characteristicsof the movable object and/or the payload. For example, the user terminalcan be used to control the movable object such that the movable objectis able to navigate to a target area. The user terminal may be used togive the movable object instructions or commands that are transmitted tothe movable object (e.g., a flight controller of the movable object)that effects autonomous flight of the movable object as furtherdescribed herein. In some instances, the user terminal may be used tomanually control the movable object and/or modify parameters of themovable object while the movable object is autonomously operating.

The movable object 100 may be any object capable of traversing anenvironment. The movable object may be capable of traversing air, water,land, and/or space. The environment may include objects that areincapable of motion (stationary objects) and objects that are capable ofmotion. Examples of stationary objects may include geographic features,plants, landmarks, buildings, monolithic structures, or any fixedstructures. Examples of objects that are capable of motion includepeople, vehicles, animals, projectiles, etc.

In some cases, the environment may be an inertial reference frame. Theinertial reference frame may be used to describe time and spacehomogeneously, isotropically, and in a time-independent manner. Theinertial reference frame may be established relative to the movableobject, and move in accordance with the movable object. Measurements inthe inertial reference frame can be converted to measurements in anotherreference frame (e.g., a global reference frame) by a transformation(e.g., Galilean transformation in Newtonian physics).

The movable object 100 may be a vehicle. The vehicle may be aself-propelled vehicle. The vehicle may traverse an environment with aidof one or more propulsion units 107. The vehicle may be an aerialvehicle, a land-based vehicle, a water-based vehicle, or a space-basedvehicle. The vehicle may be an unmanned vehicle. The vehicle may becapable of traversing an environment without a human passenger onboard.Alternatively, the vehicle may carry a human passenger. In someembodiments, the movable object may be an unmanned aerial vehicle (UAV).

Any description herein of a UAV or any other type of movable object mayapply to any other type of movable object or various categories ofmovable objects in general, or vice versa. For instance, any descriptionherein of a UAV may apply to any unmanned land-bound, water-based, orspace-based vehicle. Further examples of movable objects are provided ingreater detail elsewhere herein.

As mentioned above, the movable object may be capable of traversing anenvironment. The movable object may be capable of flight within threedimensions. The movable object may be capable of spatial translationalong one, two, or three axes. The one, two or three axes may beorthogonal to one another. The axes may be along a pitch, yaw, and/orroll axis. The movable object may be capable of rotation about one, two,or three axes. The one, two, or three axes may be orthogonal to oneanother. The axes may be a pitch, yaw, and/or roll axis. The movableobject may be capable of movement along up to 6 degrees of freedom. Themovable object may include one or more propulsion units that may aid themovable object in movement. For instance, the movable object may be aUAV with one, two or more propulsion units. The propulsion units may beconfigured to generate lift for the UAV. The propulsion units mayinclude rotors. The movable object may be a multi-rotor UAV.

The movable object may have any physical configuration. For instance,the movable object may have a central body with one or arms or branchesextending from the central body. The arms may extend laterally orradially from the central body. The arms may be movable relative to thecentral body or may be stationary relative to the central body. The armsmay support one or more propulsion units. For instance, each arm maysupport one, two or more propulsion units.

The movable object may have a housing. The housing may be formed from asingle integral piece, two integral pieces, or multiple pieces. Thehousing may include a cavity within where one or more components aredisposed. The components may be electrical components, such as a motioncontroller (e.g., a flight controller), one or more processors, one ormore memory storage units, one or more sensors (e.g., one or moreinertial sensors or any other type of sensor described elsewhereherein), one or more navigational units (e.g., a global positioningsystem (GPS) unit), one or communication units, or any other type ofcomponent. The housing may have a single cavity or multiple cavities. Insome instances, a motion controller (such as a flight controller) may bein communication with one or more propulsion units and/or may controloperation of the one or more propulsion units. The motion controller (orflight controller) may communicate and/or control operation of the oneor more propulsion units with aid of one or more electronic speedcontrol (ESC) modules. The motion controller (or flight controller) maycommunicate with the ESC modules to control operation of the propulsionunits.

The movable object may support an on-board payload 104. The payload mayhave a fixed position relative to the movable object, or may be movablerelative to the movable object. The payload may spatially translaterelative to the movable object. For instance, the payload may move alongone, two or three axes relative to the movable object. The payload mayrotate relative to the movable object. For instance, the payload mayrotate about one, two or three axes relative to the movable object. Theaxes may be orthogonal to on another. The axes may be a pitch, yaw,and/or roll axis. Alternatively, the payload may be fixed or integratedinto the movable object.

The payload may be movable relative to the movable object with aid of acarrier 102. The carrier may include one or more gimbal stages that maypermit movement of the carrier relative to the movable object. Forinstance, the carrier may include a first gimbal stage that may permitrotation of the carrier relative to the movable object about a firstaxis, a second gimbal stage that may permit rotation of the carrierrelative to the movable object about a second axis, and/or a thirdgimbal stage that may permit rotation of the carrier relative to themovable object about a third axis. Any descriptions and/orcharacteristics of carriers as described elsewhere herein may apply.

The payload may include a device capable of sensing the environmentabout the movable object, a device capable of emitting a signal into theenvironment, and/or a device capable of interacting with theenvironment.

One or more sensors may be provided as a payload, and may be capable ofsensing the environment. The one or more sensors may include an imagingdevice. An imaging device may be a physical imaging device. An imagingdevice can be configured to detect electromagnetic radiation (e.g.,visible, infrared, and/or ultraviolet light) and generate image databased on the detected electromagnetic radiation. An imaging device mayinclude a charge-coupled device (CCD) sensor or a complementarymetal-oxide-semiconductor (CMOS) sensor that generates electricalsignals in response to wavelengths of light. The resultant electricalsignals can be processed to produce image data. The image data generatedby an imaging device can include one or more images, which may be staticimages (e.g., photographs), dynamic images (e.g., video), or suitablecombinations thereof. The image data can be polychromatic (e.g., RGB,CMYK, HSV) or monochromatic (e.g., grayscale, black-and-white, sepia).The imaging device may include a lens configured to direct light onto animage sensor.

The imaging device can be a camera. A camera can be a movie or videocamera that captures dynamic image data (e.g., video). A camera can be astill camera that captures static images (e.g., photographs). A cameramay capture both dynamic image data and static images. A camera mayswitch between capturing dynamic image data and static images. Althoughcertain embodiments provided herein are described in the context ofcameras, it shall be understood that the present disclosure can beapplied to any suitable imaging device, and any description hereinrelating to cameras can also be applied to any suitable imaging device,and any description herein relating to cameras can also be applied toother types of imaging devices. A camera can be used to generate 2Dimages of a 3D scene (e.g., an environment, one or more objects, etc.).The images generated by the camera can represent the projection of the3D scene onto a 2D image plane. Accordingly, each point in the 2D imagecorresponds to a 3D spatial coordinate in the scene. The camera maycomprise optical elements (e.g., lens, mirrors, filters, etc). Thecamera may capture color images, greyscale image, infrared images, andthe like. The camera may be a thermal imaging device when it isconfigured to capture infrared images.

In some embodiments, the payload may include multiple imaging devices,or an imaging device with multiple lenses and/or image sensors. Thepayload may be capable of taking multiple images substantiallysimultaneously. The multiple images may aid in the creation of a 3Dscene, a 3D virtual environment, a 3D map, or a 3D model. For instance,a right image and a left image may be taken and used for stereo-mapping.A depth map may be calculated from a calibrated binocular image. Anynumber of images (e.g., 2 or more, 3 or more, 4 or more, 5 or more, 6 ormore, 7 or more, 8 or more, 9 or more) may be taken simultaneously toaid in the creation of a 3D scene/virtual environment/model, and/or fordepth mapping. The images may be directed in substantially the samedirection or may be directed in slightly different directions. In someinstances, data from other sensors (e.g., ultrasonic data, LIDAR data,data from any other sensors as described elsewhere herein, or data fromexternal devices) may aid in the creation of a 2D or 3D image or map.

The imaging device may capture an image or a sequence of images at aspecific image resolution. In some embodiments, the image resolution maybe defined by the number of pixels in an image. In some embodiments, theimage resolution may be greater than or equal to about 352×420 pixels,480×320 pixels, 720×480 pixels, 1280×720 pixels, 1440×1080 pixels,1920×1080 pixels, 2048×1080 pixels, 3840×2160 pixels, 4096×2160 pixels,7680×4320 pixels, or 15360×8640 pixels. In some embodiments, the cameramay be a 4K camera or a camera with a higher resolution.

The imaging device may capture a sequence of images at a specificcapture rate. In some embodiments, the sequence of images may becaptured standard video frame rates such as about 24 p, 25 p, 30 p, 48p, 50 p, 60 p, 72 p, 90 p, 100 p, 120 p, 300 p, 50 i, or 60 i. In someembodiments, the sequence of images may be captured at a rate less thanor equal to about one image every 0.0001 seconds, 0.0002 seconds, 0.0005seconds, 0.001 seconds, 0.002 seconds, 0.005 seconds, 0.01 seconds, 0.02seconds, 0.05 seconds. 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second,2 seconds, 5 seconds, or 10 seconds. In some embodiments, the capturerate may change depending on user input and/or external conditions (e.g.rain, snow, wind, unobvious surface texture of environment).

The imaging device may have adjustable parameters. Under differingparameters, different images may be captured by the imaging device whilesubject to identical external conditions (e.g., location, lighting). Theadjustable parameter may comprise exposure (e.g., exposure time, shutterspeed, aperture, film speed), gain, gamma, area of interest,binning/subsampling, pixel clock, offset, triggering, ISO, etc.Parameters related to exposure may control the amount of light thatreaches an image sensor in the imaging device. For example, shutterspeed may control the amount of time light reaches an image sensor andaperture may control the amount of light that reaches the image sensorin a given time. Parameters related to gain may control theamplification of a signal from the optical sensor. ISO may control thelevel of sensitivity of the camera to available light.

In some alternative embodiments, an imaging device may extend beyond aphysical imaging device. For example, an imaging device may include anytechnique that is capable of capturing and/or generating images or videoframes. In some embodiments, the imaging device may refer to analgorithm that is capable of processing images obtained from anotherphysical device.

A payload may include one or more types of sensors. Some examples oftypes of sensors may include location sensors (e.g., global positioningsystem (GPS) sensors, mobile device transmitters enabling locationtriangulation), motion sensors, vision sensors (e.g., imaging devicescapable of detecting visible, infrared, or ultraviolet light, such ascameras), proximity or range sensors (e.g., ultrasonic sensors, lidar,time-of-flight or depth cameras), inertial sensors (e.g.,accelerometers, gyroscopes, and/or gravity detection sensors, which mayform inertial measurement units (IMUs)), altitude sensors, attitudesensors (e.g., compasses), pressure sensors (e.g., barometers),temperature sensors, humidity sensors, vibration sensors, audio sensors(e.g., microphones), and/or field sensors (e.g., magnetometers,electromagnetic sensors, radio sensors).

The sensing data provided by the sensors may be used to control thespatial disposition, velocity, and/or orientation of the movable object(e.g., using a suitable processing unit and/or control module, asdescribed below). Alternatively, the sensors may be used to provide dataregarding the environment surrounding the movable object, such asweather conditions, proximity to potential obstacles, location ofgeographical features, location of manmade structures, and the like.

The payload may include one or more devices capable of emitting a signalinto an environment. For instance, the payload may include an emitteralong an electromagnetic spectrum (e.g., visible light emitter,ultraviolet emitter, infrared emitter). The payload may include a laseror any other type of electromagnetic emitter. The payload may emit oneor more vibrations, such as ultrasonic signals. The payload may emitaudible sounds (e.g., from a speaker). The payload may emit wirelesssignals, such as radio signals or other types of signals.

The payload may be capable of interacting with the environment. Forinstance, the payload may include a robotic arm. The payload may includean item for delivery, such as a liquid, gas, and/or solid component. Forexample, the payload may include pesticides, water, fertilizer,fire-repellant materials, food, packages, or any other item.

Any examples herein of payloads may apply to devices that may be carriedby the movable object or that may be part of the movable object. Forinstance, one or more sensors 108 may be part of the movable object. Theone or more sensors may or may be provided in addition to the payload.This may apply for any type of payload, such as those described herein.

The movable object may be capable of communicating with the userterminal 106. The user terminal may communicate with the movable objectself, with a payload of the movable object, and/or with a carrier of themovable object, wherein the carrier is used to support the payload. Anydescription herein of communications with the movable object may alsoapply to communications with the payload of the movable object, thecarrier of the movable object, and/or one or more individual componentsof the movable object (e.g., communication unit, navigation unit,propulsion units, power source, processors, memory storage units, and/oractuators).

The communications between the movable object and the user terminal maybe via wireless communications 116. For example, a communication system110 may be provided on the movable object. A corresponding communicationunit 114 may be provided on the user terminal and may be used to form acommunication link (e.g., wireless communication link) between thecommunication systems. Direct communications may be provided between themovable object and the user terminal. The direct communications mayoccur without requiring any intermediary device or network. Indirectcommunications may be provided between the movable object and the userterminal. The indirect communications may occur with aid of one or moreintermediary device or network. For instance, indirect communicationsmay utilize a telecommunications network. Indirect communications may beperformed with aid of one or more router, communication tower,satellite, or any other intermediary device or network. Examples oftypes of communications may include, but are not limited to:communications via the Internet, Local Area Networks (LANs), Wide AreaNetworks (WANs), Bluetooth, Near Field Communication (NFC) technologies,networks based on mobile data protocols such as General Packet RadioServices (GPRS), GSM, Enhanced Data. GSM Environment (EDGE), 3G, 4G, orLong Term Evolution (LTE) protocols, Infra-Red (IR) communicationtechnologies, and/or Wi-Fi, and may be wireless, wired, or a combinationthereof.

The user terminal may be any type of external device. The user terminalmay individually, or collectively refer to a device configured toreceive input from a user. The user terminal may configure one or moreuser interfaces configured to receive user inputs. Examples of userterminals may include, but are not limited to, smartphones/cellphones,tablets, personal digital assistants (PDAs), laptop computers, desktopcomputers, media content players, video gaming station/system, virtualreality systems, augmented reality systems, wearable devices (e.g.,watches, glasses, gloves, headgear (such as hats, helmets, virtualreality headsets, augmented reality headsets, head-mounted devices(MID), headbands), pendants, armbands, leg bands, shoes, vests),gesture-recognition devices, microphones, any electronic device capableof providing or rendering image data, remote controllers with controlsticks, or any other type of device. The user terminal may be a handheldobject. The user terminal may be portable. The user terminal may becarried by a human user. In some cases, the user terminal may be locatedremotely from a human user, and the user can control the user terminalusing wireless and/or wired communications. Various examples, and/orcharacteristics of user terminals are provided in greater detailelsewhere herein.

The user terminals may include one or more processors that may becapable of executing non-transitory computer readable media that mayprovide instructions for one or more actions. The user terminals mayinclude one or more memory storage devices comprising non-transitorycomputer readable media including code, logic, or instructions forperforming the one or more actions. The user terminal may includesoftware applications that allow the user terminal to communicate withand receive imaging data from a movable object. The user terminals mayinclude a communication unit 114, which may permit the communicationswith the movable object. In some instances, the communication unit mayinclude a single communication module, or multiple communicationmodules. In some instances, the user terminal may be capable ofinteracting with the movable object using a single communication link ormultiple different types of communication links. The user terminal maybe used to control movement of the movable object. In some instances,the user terminal may be configured to effect autonomous operation(e.g., autonomous flight) of the movable device, e.g., in response to auser input. In some instances, the user terminal may be configured toeffect and/or modify the autonomous operation of the movable device, asfurther described below. In some instances, the user terminal mayoptionally be used to control any component of the movable object (e.g.,operation of the payload, operation of the carrier, one or more sensors,communications, navigation, landing stand, actuation of one or morecomponents, power supply control, or any other function).

The user terminal may comprise one or more user interfaces which may beprovided on one or more devices. For example the user terminal maycomprise one, two, three, four, five, six, seven, eight, nine, ten, ormore user interfaces. A user interface may refer to an interface whereinputs by a user (e.g., an operator of the UAV) is received. The inputmay be of any type. For example, the user may provide an input by simplytouching a portion (e.g., capacitive touchscreen) of the user interface.For example, the user may actuate a mechanism keyboard, mouse, button,joysticks, etc) on the user interface to provide the user input. In someinstances, the user may provide an auditory signal (e.g., voice command)to the user interface which is received by the user interface. In someinstances, the user interface may be configured to sense, follow, ortrack movement of the user (e.g., eye movement, hand gesture, etc) inorder to receive the user input. In some instances the user interfacemay be configured to receive differing degrees of user input. Forexample, the user may exert differing amounts of force or actuatemechanisms on the user interface to differing degrees which may beappropriately interpreted by the user interface (or one or moreprocessors coupled to the user interface). For example, the user mayprovide input for differing amounts of duration which may beappropriately interpreted by the user interface ne or more processorscoupled to the user interface). Alternatively or in addition, the userinput may be configured to receive and interpret user inputs as a binaryinput. For example, a user touching the user interface may beinterpreted as a command effect flight of the movable object towards atarget. Each of the user interfaces may be provided on a separatedevice. Alternatively, two, three, four, five, or more user interfacesmay be provided on a single device.

In some instances, different user interfaces may be configured tocontrol different functionalities of the movable object and/or tocontrol different components of the movable objects. For example, afirst user terminal may be used for effecting autonomous operation ofthe movable object while a second user terminal may be used formodifying (e.g., effecting) the autonomous operation. In some instances,different devices may be configured to control different functionalitiesof the movable object and/or to control different components of themovable objects. The different user interfaces and/or devices may or maynot be in communication with one another. For example, different devicesmay be in communication with one another via wireless or wiredcommunication links. Alternatively, each of the different devices maynot be in communication with one another but separately communicate withthe UAV.

FIG. 2 illustrates a user terminal 200, in accordance with embodiments.The user terminal may comprise one or more user interfaces. Userinterface may be configured to receive inputs by a user (e.g., anoperator of the UAV). The user may be an operator of the movable object.Each of the user interfaces may be provided on a single device.Alternatively, different user interfaces may be provided on differentdevices, and the user ten ay comprise two or more devices. The userterminal may be configured to receive user input and generate and/orprovide instructions (e.g., signals) that are sent to the movable objectand/or payload. In some instances, a single input may be received at theuser terminal that generates instructions for effecting autonomousflight of the movable object. For example, a target may be designated onthe user terminal and instructions may be generated and sent to themovable object (e.g., a flight controller of the movable object) suchthat the movable object autonomously moves towards the target. In someinstances, a user input may be received at the user terminal to affector modify the autonomous operation of the movable object. For example,the user terminal may be used to manually control movement of themovable object and/or used to modify a flight (e.g., autonomous flight)of the movable object.

In some instances, the user terminal may comprise a first user interface202 and a second user interface 204. The first user interface and thesecond user interface may have different characteristics and may be usedto implement different functionalities for the movable object. Forexample, the first user interface may be configured to receive userinput that effects autonomous operation (e.g., autonomous flight) of themovable object. In some instances, the first user interface may beconfigured to receive user inputs that instruct the movable object toautonomously accomplish a certain task or autonomously move towards atarget. A single input, or a discrete number of inputs may be sufficientto instruct the movable object to autonomously operate (e.g.,autonomously fly). Continuous monitoring or supervision of the movableobject by the user may not be necessary when utilizing the first userinterface.

The first user interface 202 may comprise a display 203. The display maybe a screen. The display may or may not be a touchscreen. The displaymay be a light-emitting diode (LED) screen, OLED screen, liquid crystaldisplay (LED) screen, plasma screen, or any other type of screen. Thedisplay may be configured to show an image. The image on the display mayshow a view collected with aid of a payload of the movable object. Forinstance, an image collected by the imaging device may be shown on thedisplay. In some instances, the image collected by the imaging devicemay be considered a first person view (FPV) image. In some instances, asingle imaging device may be provided and a single FPV may be provided.Alternatively, multiple imaging devices having different fields of viewmay be provided. The views may be toggled between the multiple FPVs, orthe multiple FPVs may be shown simultaneously. The multiple FPVs maycorrespond to (or generated by) different imaging devices, which mayhave different field of views.

In another example, the image on the display may show a map that may begenerated with aid of information from a payload of the movable object.The map may optionally be generated with aid of multiple imaging devices(e.g., right camera, left camera, or more cameras), which may utilizestereo-mapping techniques. In some instances, the map may be generatedbased on positional information about the movable object relative to theenvironment, the imaging device relative to the environment, and/or themovable object relative to the imaging device. Positional informationmay include posture information, spatial location information, angularvelocity, linear velocity, angular acceleration, and/or linearacceleration. The map may be optionally generated with aid of one ormore additional sensors, as described in greater detail elsewhereherein. The map may be a two-dimensional map or a three-dimensional map.In some instances, the two-dimensional map may show a top-down view map.In some instances, the two-dimensional map may be a topographic map thatshows various natural and man-made features. In some instances, thetwo-dimensional snap may be a profile map (e.g., showing an elevation).In some instances, the views may be toggled (e.g., between thetopographical map and profile map). In some instances, the views may betoggled between a two-dimensional and a three-dimensional map view, orthe two-dimensional and three-dimensional map views may be shownsimultaneously. The views may be toggled between one or more FPV and oneor more map view, or the one or more FPV and one or more map view may beshown simultaneously.

In some embodiments, the image may be provided in a 3D virtualenvironment that is displayed on the first user interface (e.g., virtualreality system or augmented reality system). The 3D virtual environmentmay optionally correspond to a 3D map. The virtual environment maycomprise a plurality of points or objects that can be manipulated by auser. The user can manipulate the points or objects through a variety ofdifferent actions in the virtual environment. Examples of those actionsmay include selecting one or more points or objects, drag-and-drop,translate, rotate, spin, push, pull, zoom-in, zoom-out, etc. Any type ofmovement action of the points or objects in a three-dimensional virtualspace may be contemplated.

The first user interface may comprise a graphical user interface (GUI).The GUI may show an image that may permit a user to control actions ofthe movable object. The GUI may show an image that permits a user toinstruct the movable object to autonomously operate or accomplish giventasks. For example, the user may be able to select a target to track,select an area predetermined area or a point of interest) to navigatetowards, select one or more waypoints that the movable object is tonavigate across, have the movable object return to the user (e.g., userterminal), etc. In some instances, the user may be able to instruct themovable object to accomplish the tasks simply by touching or tapping apoint (e.g., portion) on the first user interface. In some instances,the first user interface may comprise a capacitive touch screen. Thefirst user interface may be enable a tap and go function for the movableobject. By simply tapping on the first user interface (e.g., a desiredlocation on a map displayed on the first user interface), the movableobject may be instructed to autonomously operate towards the tappedobject and/or area. For example, for an image on the display that showsa view collected with aid of a payload of the movable object, a usertapping on an object of interest may instruct the movable object toautonomously follow, or track the object. For example, for an image onthe display that shows a map (e.g., 2D or 3D map), a user tapping on alocation on the map may instruct the movable object to autonomouslynavigate towards the tapped location.

In some instances, a target may be selected by a user at the first userinterface. The target may be selected within the image (e.g., of thedisplay). The user may also select a portion of the image (e.g., point,region, and/or object) to define the target and/or direction. The usermay select the target by directly touching the screen (e.g.,touchscreen). The user may touch a portion of the screen. The user maytouch the portion of the screen by touching a point on the screen. Theuser may select the target by selecting the portion of the image withaid of a user interactive device (e.g., mouse, joystick, keyboard,trackball, touchpad, button, verbal commands, gesture-recognition,attitude sensor, thermal sensor, touch-capacitive sensors, or any otherdevice). A touchscreen may be configured to detect location of theuser's touch, length of touch, pressure of touch, and/or touch motion,whereby each of the aforementioned manner of touch may be indicative ofa specific input command from the user.

The movable object may be configured to travel toward, navigate around,and/or visually track the target. The target may be a targetdestination. In some instances, the target destination may be locationsselected on an image (e.g., FPV image) captured by an imaging device onboard the movable object. For example, a location (or locations) may beselected by a user on an image (e.g., FPV image) captured by an imagingdevice, e.g., by touching points on the image. Such tapping on portionsof the image may instruct the movable object to fly to the location witha flight path. In some instances, the target destination may belocations selected on a map. For example, locations 206, 208, and 210may comprise targets that have been selected by a user on a map. Thetarget may be selected for example, by touching points on the mapsubstantially as described above. Such tapping on portions of the mapmay instruct the movable object to fly (e.g., autonomously) to thetarget with a flight path. Such tapping on portions of the userinterface to instruct the movable object to autonomously fly towards thetarget (e.g., locations or objects) may herein be referred to as a tapto go function. In some instances, the target destination may bepre-determined or pre-configured destinations selected without aid of animage or a map, e.g., from a predetermined list, as a standalonefeature, etc. For example, the target destination may be locations ofthe user terminal, location of the user, designated way points, orpoints of interest (e.g., home as designated by the user).

In some instances, the target may be a target object. The target objectmay be a stationary target or a moving target. In some instances, theuser may specify whether the target is a stationary or moving target.Alternatively, the user may provide any other type of indicator ofwhether the target is a stationary or moving target. Alternatively, noindication may be provided, and a determination may be automaticallymade with aid of one or more processors, optionally without requiringuser input whether the target is a stationary target or a moving target.A target object may be classified as a stationary target or a movingtarget depending on its state of motion. In some cases, a target objectmay be moving or stationary at any given point in time. When the targetobject is moving, the target object may be classified as a movingtarget. Conversely, when the same target object is stationary, thetarget object may be classified as a stationary target.

A stationary target may remain substantially stationary within anenvironment. Examples of stationary targets may include, but are notlimited to landscape features e.g., trees, plants, mountains, hills,rivers, streams, creeks, valleys, boulders, rocks, etc.) or manmadefeatures (e.g., structures, buildings, roads, bridges, poles, fences,unmoving vehicles, signs, lights, etc.). Stationary targets may includelarge targets or small targets. A user may select a stationary target.The stationary target may be recognized. Optionally, the stationarytarget may be mapped. The movable object may travel to and/or navigatearound the stationary target, and/or track the stationary object. Insome instances, the stationary target may correspond to a selectedportion of a structure or object. For example, the stationary target maycorrespond to a particular section (e.g., top floor) of a skyscraper.

A moving target may be capable of moving within the environment. Themoving target may always be in motion, or may be at motions for portionsof a time. The moving target may move in a fairly steady direction ormay change direction. The moving target may move in the air, air land,underground, on or in the water, and/or in space. The moving target maybe a living moving target (e.g., human, animal) or a non-living movingtarget (e.g., moving vehicle, moving machinery, object blowing in windor carried by water, object carried by living target). The moving targetmay include a single moving object or a group of moving objects. Forinstance, the moving target may include a single human or a group ofmoving humans. Moving targets may be large targets or small targets. Auser may select a moving target. The moving target may be recognized.Optionally, the moving target may be mapped. The movable object maytravel to and/or navigate around the moving target and/or track themoving object. A flight path may be planned for the movable object tonavigate around the moving object. The path may be changed or updated asthe moving object moves along the path. Alternatively, the movableobject may travel to and/or navigate around the stationary object and/orvisually track the moving object without requiring a planned path.

A moving target may be any object configured to move within any suitableenvironment, such as in air (e.g., a fixed-wing aircraft, a rotary-wingaircraft, or an aircraft having neither fixed wings nor rotary wings),in water (e.g., a ship or a submarine), on ground (e.g., a motorvehicle, such as a car, truck, bus, van, motorcycle; a movable structureor frame such as a stick, fishing pole; or a train), under the ground(e.g., a subway), in space (e.g., a spaceplane, a satellite, or aprobe), or any combination of these environments.

A moving target may be capable of moving freely within the environmentwith respect to six degrees of freedom (e.g., three degrees of freedomin translation and three degrees of freedom in rotation). Alternatively,the movement of the moving target can be constrained with respect to oneor more degrees of freedom, such as by a predetermined path, track, ororientation. The movement can be actuated by any suitable actuationmechanism, such as an engine or a motor. The actuation mechanism of themoving target can be powered by any suitable energy source, such aselectrical energy, magnetic energy, solar energy, wind energy,gravitational energy, chemical energy, nuclear energy, or any suitablecombination thereof. The moving target may be self-propelled via apropulsion system, such as described further below. The propulsionsystem may optionally run on an energy source, such as electricalenergy, magnetic energy, solar energy, wind energy, gravitationalenergy, chemical energy, nuclear energy, or any suitable combinationthereof.

In some instances, the moving target can be a vehicle, such as aremotely controlled vehicle. Suitable vehicles may include watervehicles, aerial vehicles, space vehicles, or ground vehicles. Forexample, aerial vehicles may be fixed-wing aircraft (e.g., airplane,gliders), rotary-wing aircraft (e.g., helicopters, rotorcraft), aircrafthaving both fixed wings and rotary wings, or aircraft having neither(e.g., blimps, hot air balloons). A vehicle can be self-propelled, suchas self-propelled through the air, on or in water, in space, or on orunder the ground. A self-propelled vehicle can utilize a propulsionsystem, such as a propulsion system including one or more engines,motors, wheels, axles, magnets, rotors, propellers, blades, nozzles, orany suitable combination thereof. In some instances, the propulsionsystem can be used to enable the movable object to take off from asurface, land on a surface, maintain its current position and/ororientation (e.g., hover), change orientation, and/or change position.

Selecting a target as described above may effect autonomous flight ofthe movable object. For example, selecting a target may generateinstructions that are sent to a flight controller of the movable object,e.g., via the communications systems. The flight controller may receivethe instructions and further generate signals to effect autonomousflight of the movable object. The autonomous flight may be autonomousflight towards the target. As described herein, the target may be atarget destination (e.g., location) and/or target object. In someinstances, a plurality of targets may be selected and the movable objectmay fly along the targets. A movable object under autonomous operation(e.g., via the received input at the first user interface) may comprisea predetermined flight speed at which it moves. The predetermined flightspeed may be a default speed. In some instances, the predeterminedflight speed may be user configurable. In some instances, thepredetermined flight speed may be equal to or less than about 2 m/s, 4m/s, 6 m/s, 8 m/s, 10 m/s, 12 m/s, 15 m/s, 20 m/s, or 50 m/s.

An object under autonomous operation (e.g., via the received input atthe first user interface) may comprise a trajectory, or a flight path.An autonomous flight may comprise an autonomous flight path for themovable object. In some cases, a flight path 205 of the autonomousflight may be displayed in the GUI Alternatively or in addition, aplurality of points 206, 208, 210 that are indicative of targets towardswhich the movable object is autonomously flying towards may be displayedin the GUI. The targets may indicate a target object and/or target areasthe movable object is autonomously flying towards. In some instances,the flight path may comprise a preset direction, preset trajectory,autonomously planned trajectory, and/or a user configured trajectory. Insome instances, the flight path may be preset (e.g., take the shortestroute at a certain altitude). In some instances, the flight path may beselected by the user (e.g., from a number of different pre-configuredflight paths). In some instances, a user may generate a flight path forthe movable object by drawing a contour on the screen, e.g., with a userinteractive device or user appendage, substantially as described above.In some instances, the flight path may be generated autonomously orsemi-autonomously. In some instances, the flight path may be generatedrelative to a target by taking into account a position, orientation,attitude, size, shape, and/or geometry of the target. In some instances,the flight path may be generated autonomously or semi-autonomouslytaking into account other parameters such as parameters of the movableobject (e.g., size, weight, velocity, etc.), jurisdictional parameters(e.g., laws and regulations), or environmental parameters (e.g., windconditions, visibility, obstacles, etc). In some instances, the user maymodify any portion of the flight path by adjusting (e.g., moving)different spatial points of the motion path on the screen (e.g., on thefirst user interface). Alternatively, the user may select a region on ascreen from a pre-existing set of regions, or may draw a boundary for aregion, a diameter of a region, or specify a portion of the screen inany other way.

The movable object may travel along the flight path until acountermanding instruction is received or when a countermandingcondition is realized. For instance, the movable object mayautomatically travel along the motion path until a new path is input,when a portion of the motion path is changed, or when a new target isinput. The movable object may travel along the flight path until adifferent flight path is selected. In some instances, the user may takemanual control over the motion of the movable object at any time whileit is moving.

The user terminal may optionally comprise a second user interface 204.In some instances, the second user interface may be different from thefirst user interface. In some instances, the second user interface maybe of a different type than the first user interface and/or may beconfigured to receive different modes of user input. In some instances,the second user interface may comprise one or more mechanisms 212, 214where user input is received. The one or more mechanisms may be capableof being actuated. The one or more mechanisms may include any type ofhardware mechanism such as control sticks, physical buttons, or scrollwheels. In some instances, the one or more mechanisms may includesoftware mechanisms, e.g., interactive buttons on a touch screen. Whilecontrol sticks are primarily described herein, it is to be understoodthat the use of other mechanisms (e.g., buttons, etc) may be equallyapplicable.

Control sticks may also be referred to as joy sticks (or joysticks). Insome instances, the one or more control sticks may comprise a roll stickconfigured to affect rotation of the UAV about a roll axis and/or a yawstick configured to affect a rotation of the UAV about a yaw axis. Insome instances, the one or more control sticks may comprise a pitchstick. The pitch stick may be configured to affect change in a velocityof the UAV. In some instances, the one or more control sticks maycomprise a throttle stick. The throttle stick may be configured toaffect a change in a height altitude) of the UAV. In some instances, thesecond user interface may be used to control movement of the movableobject. The second user interface may be used to directly control motionof the movable object. Alternatively or in addition, the second userinterface may be used to modify movement (e.g., flight) of the movableobject that is under autonomous control. In some instances, the seconduser interface may be used to effect autonomous flight of the UAV.

While control sticks may be designated with certain names (e.g., pitchstick, yaw stick, etc), it is to be understood that the designations ofthe control sticks are arbitrary. For example, the user terminal (e.g.,the second user interface) may be able operate under different modes.For example, the user terminal may operate under different modes with agiven command from a user, e.g., actuation of a switch). Under differentmodes, a control stick (e.g., control stick 212 or 214) may beconfigured to affect operation of the UAV in different ways. In someinstances, in one operating mode, an actuation mechanism may beconfigured to effect autonomous flight flight along a predetermineddirection or flight along a previously headed direction) while inanother operating mode, an actuation mechanism may be configured toaffect the flight of the UAV under autonomous flight.

In some instances, in a first mode, control stick 212 may be configuredto affect a forward and backward movement of the UAV while in a secondmode, control stick 212 may be configured to affect a velocity of theUAV moving in a forward direction. In a third operating mode, thecontrol stick 212 may be configured to affect a height of the UAV and/ora rotation of the UAV about one or more axes. The user terminal maycomprise one, two, three, four, five or more operating modes. Inaddition, a given control stick (e.g., control stick 212 or 214) maycomprise more than one functionality, or may affect a flight (e.g.,autonomous flight) of the UAV in more than one parameter. For example,control stick 212 moving forward and backward may affect a change inheight of the of a UAV while the control stick 212 moving left and rightmay affect rotation of a UAV about a roll axis.

In some instances, the second user interface may be utilized to effectreal time control of the movable object. In some instances, the firstuser interface and the second user interface may work together inconcert. FIG. 3 illustrates a first user interface 302 and a second userinterface 304 working in concert, in accordance with embodiments. Thefirst user interface may be as previously described. For example, thefirst user interface may comprise a display configured to show one ormore images. For example, the display may be configured to show an imageof a map 306. The map may be a two dimensional or three dimensional mapof an environment around the movable object. Alternatively or inaddition, the display may be configured to show a first person viewimage 308 acquired by a payload coupled to the movable object. Forexample, the first person view image of FIG. 3 shows an obstacle 310which the movable object is heading towards.

The second user interface may comprise one or more control sticks 314,316, The control sticks may be utilized to affect parameters of themovable object, e.g., in real time. In some instances, the controlsticks may affect and/or modify autonomous operation (e.g., autonomousflight) of the movable object. For example, a first user input may bereceived at the first user interface. The first user input may effectautonomous flight of the movable object. For example, a user tapping atarget on the map 306 may generate instructions that are sent to aflight controller of the movable object that effects autonomous flighttowards the target. The autonomously operating movable object maycomprise a flight path 312. While autonomously navigating towards thetarget, the first user interface may show a first person view image 308of images captured by a payload coupled to the movable object. Inputreceived at the second user interface may subsequently affect or modifythe autonomous operation of the movable object. In some instances, inputreceived at the second user interface may disrupt the autonomous flightof the movable object. Disrupting autonomous flight via the second userinterface may provide an effective and easy way to quickly disruptautonomous flight, e.g., in emergencies or unanticipated situationswhere interaction with the first user interface (e.g., GUI) isundesired. In some instances, input received at the second userinterface may modify autonomous flight of the movable object withoutdisrupting the autonomous operation. For example, the flight path of themovable object may be modified due to the input received at the seconduser input, but the movable object may continue navigating towards thedesignated target, e.g., while the second user input is being receivedand/or after the second user input is received. For example, despiteuser input on the second user interface, autonomous operation, or flightof the movable object may be maintained such that the movable objectcontinues towards accomplishing its task (e.g., tracking a target,navigating towards a desired location, etc). In exemplary embodiments,the second user input may modify a flight path, or trajectory of themovable object. For example, input at the second user interface may adda directional component to the autonomous flight path of the movableobject or may modify the autonomous flight path by adding a velocity oracceleration component to the movable object.

This feature may be advantageous for providing user input inunanticipated situations without disrupting autonomous flight of themovable object. In some instances, the differentiation between the userinterfaces (e.g., first and second user interfaces) may proveadvantageous as the second user interface may provide an intuitive andeasy to use control scheme for modifying the autonomous flight withoutdisrupting an overall autonomous flight of the movable object. Forexample, this may be desirable in emergency situations or circumstancesunaccounted for or undetected by the movable object, where quick userinput is necessary, but where continued autonomous flight is desiredafterwards. There may be seamless transition between the autonomousflight of the movable object and modification of the autonomous flightby a second user input (e.g., in real time). In addition, there may beseamless transition between the time the movable object takes intoaccount the second user input and the time the movable object returns toautonomous flight (e.g., after the second user input).

For example, the movable object operating under autonomous control mayencounter an obstacle 310. The movable object may fail to detect theobstacle (e.g., through error, due to lack of obstacle sensors, etc)and/or fail to implement autonomous obstacle avoidance measures. In sucha case, the user may observe that there is an obstacle in a flight pathof the autonomously operating movable object. By manipulating thecontrol sticks on the second user interface, the user may be able toeasily avoid the obstacle. After avoiding the obstacle, the user mayrelease the control stick and the movable object may continueautonomously operating, or completing its task. For example, by tryingto provide input into the first user interface, the movable object maybe unable to quickly avoid the obstacle. The second user interface mayprovide a convenient interface where quick, intuitive controls may beaffected by the user. After avoiding the obstacle, the movable objectmay continue in autonomously operating, or completing its task (e.g.,tracking a target, navigating towards a destination). In exemplaryembodiments, the second user input may modify a flight path, ortrajectory of the movable object. For example, input at the second userinterface may add a directional component to the autonomous flight pathof the movable object or may modify the autonomous flight path by addinga velocity or acceleration component to the movable object.

For example, the movable object operating under autonomous controltransmit a first person view image to the first user interface. In someinstances, the user may notice an object of interest 318 in the firstperson view image. By manipulating the control sticks on the second userinterface, the user may be able to head towards (e.g., slightly modifythe trajectory of the movable object) the object of interest withoutdisrupting the autonomous operation. After the user is satisfied, theuser may release the control stick and the movable object may continueautonomously operating, or completing its task. The movable object maycontinue autonomously operating without any further input, and there maybe a seamless transition between autonomous operation and user directedmovement. In exemplary embodiments, the second user input may modify aflight path, or trajectory of the movable object. For example, input atthe second user interface may add a directional component to theautonomous flight path of the movable object or may modify theautonomous flight path by adding a velocity or acceleration component tothe movable object.

FIG. 4 illustrates a method for modifying autonomous flight of a UAV, inaccordance with embodiments. In step 401, autonomous flight of the UAVmay be effected. For example, one or more instructions may be providedin order to effect an autonomous flight of the UAV. In some instances,the one or more instructions may be provided by a user (e.g., at thefirst user interface, previously described herein). For example, theuser may provide an input at a handheld or mobile device such as a cellphone, tablet, or PDA. The handheld or mobile device may comprise atouch screen, and may be configured to display images, as previouslydescribed herein. In some instances, the images may comprise images(e.g., first person view images) received from a camera coupled to theUAV and/or images of a map (e.g., 2-D or 3-D map) that shows a locationof the UAV. By touching the handheld or mobile device, the user mayprovide the one or more instructions to effect an autonomous flight ofthe UAV.

The one or more instructions may be transmitted to a flight controllerof the UAV. In response to the one or more instructions transmitted, theflight controller may generate a first set of signals that effectsautonomous flight of the UAV, e.g., with aid of one or more processors.For example, the flight controller may generate a first set of signalsthat instructs one or more propulsion units of the UAV to operate inorder to effect the autonomous flight of the UAV.

The autonomous flight may be any flight of the UAV that does not requirecontinued input (e.g., real time input) from the user. In someinstances, the autonomous flight may have a predetermined task or goal.Examples of the predetermined task or goal may include, but are notlimited to tracking or following a target object, flying to a targetarea or a desired location, returning to a location of the user or auser terminal. In some instances, the autonomous flight may have apredetermined target that the UAV is moving towards. The target may be atarget object or a target destination. For example, the autonomousflight may be autonomous flight towards a predetermined locationindicated by the user. In some instances, the autonomous flight may beflight to a predetermined location, an autonomous return of the UAV, anautonomous navigation along one or more waypoints, autonomous flight toa point of interest.

In some instances, the autonomous flight may comprise an autonomousflight trajectory, or an autonomous flight path. In some instances, theautonomous flight may comprise an autonomous flight direction. Thetrajectory may be a flight trajectory in two, or three dimensionalcoordinates. In some instances, the autonomous flight may have a presettrajectory. For example, the preset trajectory may be to take theshortest flight path, e.g., in flying towards the target (e.g., targetdestination or target obstacle) or in accomplishes the task. In someinstances, the autonomous flight path may have an autonomously plannedtrajectory. For example, a flight controller of the UAV may calculate,or autonomously plan a trajectory taking into account variousparameters. The parameters may include, but are not limited toenvironmental conditions, regulations and laws, known obstacles, knownevents, and objectives. Based on the various parameters, the flightcontroller may set an autonomously planned trajectory best suited forthe autonomous flight. In some instances, the autonomous flight may havea user configured trajectory. For example, prior to effecting theautonomous flight, an operator of the UAV may manually configure atrajectory, or a flight path to be taken by the autonomously operatingUAV. In some instances, the trajectory or flight path of theautonomously flying UAV may be displayed on the user terminal, and/orthe handheld or mobile device which receives the user input foreffecting the autonomous flight of the UAV.

In step 403, the autonomous flight may be modified in response to a userinput. For example, during operation of the autonomous flight, a usermay provide an input at the user terminal. In some instances, the userinput may be provided via buttons or control sticks as previouslydescribed herein. The input may provide one or more instructions tomodify the autonomous flight of the UAV. The one or more instructionsmay be transmitted to a flight controller of the UAV which may generatea second set of signals that modify the autonomous flight of the UAV.For example, the flight controller may generate a second set of signalsthat further instruct one or more propulsion units to operate in orderto modify the autonomous flight of the UAV. In some instances, themodification of the autonomous flight may disrupt, or stop, theautonomous flight of the UAV, e.g., until further input. For example,the UAV whose autonomous flight has been disrupted may manually becontrolled by the user. In some instances, the UAV whose autonomousflight has been disrupted may hover at the location where the user inputhas been provided until further instructions are given. Alternatively,the UAV whose autonomous flight has been disrupted may return to theuser, or user terminal, or proceed to land.

In some instances, the autonomous flight of the UAV may be modifiedwithout disrupting the autonomous flight. For example, the UAV mayproceed in carrying out its task (e.g., tracking a target object ormoving towards a target destination) while its autonomous flight ismodified by the user input. In some instances, the UAV may proceed inflying while its flight is modified by the user input. In someinstances, the modification of the autonomous flight may affect arotation of the UAV about one or more axes of the UAV (e.g., roll axis,yaw axis, pitch axis, etc). In some instances, the modification of theautonomous flight may modify an autonomous flight path of the UAV whilemaintaining the autonomous flight. As previously described above, theability to modify a flight path (e.g., affect a trajectory) of the UAVwhile maintaining autonomous flight may be advantageous as it providesthe ability to make minor adjustments in order to deal withunanticipated situations (e.g., not accounted for by a flight controllerbut noticed by a user) without disrupting autonomous flight.Accordingly, a seamless transition between the autonomous flight of theUAV and user adjustments (e.g., in real time) may be enabled.

In some instances, the method 400 may be effected with aid of one ormore processors. For example, a system may be provided for modifyingautonomous flight of an unmanned aerial vehicle. The system may compriseone or more processors, individually or collectively configure to:effect an autonomous flight of the UAV, wherein the autonomous flightcomprises an autonomous flight path; and modify the autonomous flightpath in response to a user input, wherein e autonomous flight path ismodified while maintaining the autonomous flight.

In some instances, the method 400 may be effected with aid of anon-transitory computer readable medium comprising code, logic, orinstructions. For example, the non-transitory computer readable mediummay comprise code, logic, or instructions to effect an autonomous flightof the UAV, wherein the autonomous flight comprises an autonomous flightpath; and modify the autonomous flight path in response to a user input,wherein the autonomous flight path is modified while maintaining theautonomous flight.

In some instances, an unmanned aerial vehicle may be used to effectmethod 400. For example, the UAV may comprise a flight controllerconfigured to generate (1) a first set of signals for autonomous flightof the UAV, wherein the autonomous flight comprises an autonomous flightpath, and (2) a second set of signals for modification of the autonomousflight path, wherein the autonomous flight path is modified whilemaintaining the autonomous flight; and one or more propulsion unitsconfigured to (a) effect the autonomous flight of the UAV in response tothe first set of signals, and (b) modify the autonomous flight path ofthe UAV in response to the second set of signals.

FIG. 5 illustrates an autonomous flight of the UAV being modified by auser input, in accordance with embodiments. In some instances, a UAV maybe operating autonomously. For example, the UAV 504 may be autonomouslyflying towards a target 506, e.g., in accordance with instructions froma user. The autonomous flight may comprise an autonomous flight path508. In some instances, a user may provide an input (e.g., at the userten al) in order to modify the autonomous flight of the UAV. In someinstances, the autonomous flight path of the UAV may be modified by theuser input. For example, the user may provide an input at a remotecontroller 502 that comprises one or more control sticks 510, 512. Thecontrol sticks may be configured to affect rotation of the UAV about oneor more axes. For example, the one or more control sticks may comprise aroll stick configured to affect rotation of the UAV about a roll axisand/or a yaw stick configured to affect a rotation of the UAV about ayaw axis. In some instances, the one or more control sticks may comprisea pitch stick. The pitch stick may be configured to affect change in avelocity of the UAV. In some instances, the one or more control sticksmay comprise a throttle stick. The throttle stick may be configured toaffect a change in a height (e.g., altitude) of the UAV.

By providing an input, the user may actuate at least one of the one ormore control sticks. The user input received at the user terminal mayprovide one or more instructions to modify the autonomous flight of theUAV. The one or more instructions may be transmitted to a flightcontroller of the UAV which may generate a set of signals that modifythe autonomous flight of the UAV, e.g., by affecting a rotation of theUAV about one or more axes, by affecting a change in velocity of theUAV, or by affecting a change in a height of the UAV. For example, theflight controller may generate a set of signals that further instructone or more propulsion units to operate in order to modify theautonomous flight of the UAV, e.g., by affecting a rotation of the UAVabout one or more axes. In some instances, actuation of the roll stickmay affect rotation of the UAV about a roll axis while actuation of theyaw stick may affect rotation of the UAV about the yaw axis, e.g., whilemaintaining autonomous flight of the UAV. In some instances, actuationof the throttle stick may affect a height of the UAV while actuation ofthe pitch stick may affect a velocity of the UAV.

In some instances, the user input (e.g., actuation of the control stick)may add a direction component to the autonomous flight path 508 of theUAV. For example, the user input may modify the desired target 506 by acertain distance 514 such that the UAV is moving towards a new target516. In some instances, the UAV may move towards the new target withoutrotating about the yaw axis. For example, the UAV may move towards thenew target in connection with rotating the UAV about the roll axis. Theadded directional component may or may not be perpendicular to theautonomous flight path 508 of the UAV. In some instances, the addeddirection component may be along a reference plane. A reference plane asused herein may be any reference plane. The reference plane may be arelative reference plane that may depend on other factors. For example,the reference plane may adjust depending on a state of the UAV such as aposition and/or orientation of the UAV. For example, the reference planemay be a transverse plane of the UAV which may adjust with anorientation of the UAV. In some instances, the reference plane may be atransverse plane of the UAV in a hovering position. In some instances,the reference plane may be a transverse plane of the UAV in an uprightposition. In some instances, the reference plane may be relative toexternal factors or the environment. The reference plane may be relativeto a may be a specified or predetermined reference plane such as atransverse plane of the UAV or a horizontal plane. In some instances,the added directional component may be both perpendicular to theautonomous flight path of the UAV and along a reference plane. The addeddirectional component may be a horizontal component that modifies theautonomous flight path of the UAV in a horizontal direction. In someinstances, the added directional component may correspond to a degree ofthe user input, e.g., a duration of the user input or a force of theuser input (e.g., degree of actuation of the one or more joysticks). Forexample, if the user input is maintained, the added directionalcomponent may gradually increase. For example, a greater directionalcomponent may be added for a joystick that is actuated more than ajoystick that is less actuated. The correspondence may or may not belinear. In some instances, the added component may modify the flightpath of the UAV in accordance with any mathematical function, e.g.,linear, exponential, etc.

In some instances, the user input may add a velocity component 518 tothe UAV in modifying the autonomous flight path 520 of the UAV. Theadded velocity component may or may not be perpendicular to theautonomous flight path of the UAV. In some instances, the added velocitycomponent may be along a reference plane. For example, the addedvelocity component may be along a transverse plane of the UAV and/oralong a horizontal plane. In some instances, the added velocitycomponent may be both perpendicular to the autonomous flight path of theUAV and along a reference plane. The added velocity component may be ahorizontal component that modifies the autonomous flight path of the UAVin a horizontal direction. In some instances, the velocity component maybe added in connection with rotating the UAV about the roll axis. Insome instances, the velocity component may be added without affectingrotation of the UAV about the yaw axis. In some instances, the addedvelocity component may be continued to be applied while the user inputis maintained, as shown by the plurality of velocity components appliedover time in FIG. 5. In some instances, the added velocity component maycorrespond to a degree of the user input, e.g., a duration of the userinput or a force of the user input (e.g., degree of actuation of the oneor more joysticks). For example, if the user input is maintained, theadded velocity component may gradually increase. For example, a greatervelocity component may be added for a joystick that is actuated morethan a joystick that is less actuated. The correspondence may or may notbe linear. In some instances, the added component may modify the flightpath of the UAV in accordance with any mathematical function, e.g.,linear, exponential, etc.

In some instances, the user input may add an acceleration component 522to the UAV in modifying the autonomous flight path 524 of the UAV. Theadded acceleration component may or may not be perpendicular to theautonomous flight path of the UAV. In some instances, the addedacceleration component may be along a reference plane. For example, theadded acceleration component may be along a transverse plane of the UAVand/or along a horizontal plane. In some instances, the addedacceleration component may be both perpendicular to the autonomousflight path of the UAV and along a reference plane. The addedacceleration component may be a horizontal component that modifies theautonomous flight path of the UAV in a horizontal direction. In someinstances, the acceleration component may be added in connection withrotating the UAV about the roll axis. In some instances, theacceleration component may be added without affecting rotation of theUAV about the yaw axis. In some instances, the added accelerationcomponent may be continued to be applied while the user input ismaintained. In some instances, the added acceleration component maycorrespond to a degree of the user input, e.g., a duration of the userinput or a force of the user input (e.g., degree of actuation of the oneor more joysticks). For example, if the user input is maintained, theadded acceleration component may gradually increase. For example, agreater acceleration component may be added for a joystick that isactuated more than a joystick that is less actuated. The correspondencemay or may not be linear. In some instances, the added accelerationcomponent may modify the flight path of the UAV in accordance with anymathematical function, e.g., linear, exponential, etc.

In some instances, actuation of a roll stick may add a directioncomponent, velocity component, and/or acceleration component aspreviously described herein. For example, control stick 510 may be anexample of a roll stick. Alternatively, control stick 512 may be anexample of a roll stick. Actuation of the roll stick may add ahorizontal velocity component to the UAV. In some instances, the addedvelocity component may be due to rotation of the UAV about a roll axis.In some instances, the added velocity component may correspond to adegree of actuation of the roll stick. For example, the roll stick maycomprise a resting state where no force is exerted on the roll stick,and two fully actuated states in opposite directions. For example, theroll stick may be configured to move to the left and to the right. Insome instances, the position of the roll stick may be described as beingbetween −1 (fully actuated to the left) and 1 (full actuated to theright). For example, a roll stick that is halfway moved towards the leftmay comprise a position of −0.5 while a roll stick that is a third ofthe way moved towards the right may comprise a position of 0.333. Insome instances, a velocity component (e.g., horizontal velocitycomponent) that is added to the UAV as a result of the actuation of theroll stick may be described by the equation: (1) added velocitycomponent=(roll stick position)×velocity factor, In some instances, thevelocity factor may be a predetermined velocity value, e.g., factory setor user determined value. For example, the predetermined velocity valuemay be equal to or less than about 2 m/s, 4 m/s, 6 m/s, 8 m/s, 10 m/s,12 m/s, 15 m/s, 20 m/s, or 50 m/s. In some instances, the velocityfactor may depend on a forward velocity of the UAV. For example, theforward velocity may refer to a velocity component of the UAV along theautonomous flight path. In FIG. 5, the forward velocity component mayrefer to a velocity component of the UAV along a direction parallel toautonomous flight path 508. In some instances, the forward velocitycomponent may refer to a velocity component of the UAV along the rollaxis of the UAV. In such cases, the velocity component that is added tothe UAV as a result of the actuation of the roll stick may be describedby the equation: (2) added velocity component=(roll stickposition)×(forward velocity component)×factor. In some instances, thefactor may be equal to or less than about 0.1, 0.2, 0.4, 0.6, 0.8, 1, 2,or 4. In some instances, the factor may be equal to about 0.5.

In some instances, the user input may modify a flight path of the UAV tomake it such that the UAV flies in a curved trajectory toward adirection of the one or more control sticks. FIG. 13 illustrates a UAVflying in a curved trajectory in response to actuation of one or morecontrol sticks, in accordance with embodiments. In some instances, theuser input may affect rotation of the UAV about a yaw axis of the UAVsuch that it flies in a curved trajectory. For example, control stick1312 may be an example of a yaw stick. Alternatively, control stick 1313may be an example of a yaw stick. In some instances, a speed of the UAVflying along a preset curved trajectory (e.g., trajectory 1306, 1308,1310, etc) may depend on or be proportional to a user's input on the yawstick. Alternatively, the radius of the curve, or degree of rotation ofthe UAV about the yaw axis may inversely depend on, or be inverselyproportional to a user's input on the yaw stick. In some instances, theposition of the yaw stick may be described as being between −1 (fullyactuated to the left) and 1 (fully actuated to the right), substantiallyas described with respect to the roll stick. If the user fully actuatesthe yaw stick to the left, the UAV may fly along trajectory 1308 with asmall radius of curvature while if the user actuates the yaw stick tothe left by a small amount, the UAV may fly along trajectory 1306 with alarge radius of curvature. Similarly, if the user fully actuates the yawstick to the right, the UAV may fly along a trajectory 1310 with a smallradius of curvature.

As provided in FIGS. 5 and 13, the flight path of the autonomouslyoperating UAV may be modified intuitively. For example, actuation of theone or more control sticks 510, 512 to the left may modify the flightpath of the UAV such that it moves towards the left, e.g., whilemaintaining the autonomous flight of the UAV such that it keeps movingtowards the target. For example, actuation of the one or more controlsticks (e.g., via input from the user) to the right may modify theflight path of the UAV such that it moves towards the right.

In some instances, the added components (e.g., direction, velocity,acceleration, etc) referred to above may include vertical components.FIG. 6 illustrates a side view of an autonomous flight path of themovable object modified by a user input, in accordance with embodiments.In some instances, a UAV 604 may be autonomously flying towards a target606, e.g., in accordance with instructions from a user. The autonomousflight may comprise an autonomous flight path 608. In some instances, auser may provide an input at the user terminal) in order to modify theautonomous flight of the UAV. For example, the user may provide an inputat a ret e controller 602 that comprises one or more control sticks 610,612. In some instances, the control sticks may be configured to affect aheight of the UAV without affecting rotation of the UAV about one ormore axes of the UAV. Alternatively, the control sticks may beconfigured to affect height of the UAV via rotation of the UAV about oneor more axes.

Substantially as described above, a direction component may be added tothe autonomous flight path of the UAV. In some instances, a velocitycomponent 614 (e.g., vertical velocity component) may be added to theUAV in modifying the autonomous flight path of the UAV as shown inembodiment 620. In some instances, an acceleration component 616 (e.g.,vertical acceleration component) may be added to the UAV in modifyingthe autonomous flight path of the UAV as shown in embodiment 630. Insome instances, actuation of the one or more control sticks to the top(e.g., away from the user) may modify the flight path of the UAV suchthat it moves vertically up as shown in embodiments 620 and 630.Alternatively, actuation of the one or more control sticks to the bottommay modify the flight path of the UAV such that it moves vertically up.In some instances, actuation of the one or more control sticks to thebottom (e.g., towards the user) may modify the flight path of the UAVsuch that it moves vertically down. Alternatively, actuation of the oneor more control sticks to the top may modify the flight path of the UAVsuch that it moves vertically down.

In some instances, the control stick configured to affect a height ofthe UAV may be a throttle stick. For example, control stick 610 may be athrottle stick. Alternatively, control stick 612 may be an example of athrottle stick. Actuation of the throttle stick may add a verticalvelocity component to the UAV. In some instances, the added velocitycomponent may correspond to a degree of actuation of the throttle stick.For example, the throttle stick may comprise a resting state where noforce is exerted on the throttle stick, and two fully actuated states inopposite directions. For example, the throttle stick may be configuredto be capable of moving both (1) towards the top as indicated bydirection 609, and (2) towards the bottom. In some instances, theposition of the throttle stick may be described as being between −1(fully actuated to the bottom) and 1 (fully actuated to the top). Forexample, a throttle stick that is halfway moved towards the bottom maycomprise a position of −0.5 while a roll stick that is a third of theway moved towards the top may comprise a position of 0.333. In someinstances, a velocity component (e.g., vertical velocity component) thatis added to the UAV as a result of the actuation of the throttle stickmay be described by the equation: (1) added velocity component=(throttlestick position)×velocity factor. A negative velocity may indicate thatthe UAV is moving down while a positive velocity component may indicatethat the UAV is moving up. In some instances, the velocity factor may bea predetermined velocity value, e.g., factory set or user determinedvalue. For example, the predetermined velocity value may be equal to orless than about 2 m/s, 4 m/s, 6 m/s, 8 m/s, 10 m/s, 12 m/s, 15 m/s, 20m/s, or 50 m/s. In some instances, the predetermined velocity value maybe equal to about 3 m/s. In some instances, the velocity factor maydepend on a forward velocity of the UAV. For example, the forwardvelocity may refer to a velocity component of the UAV along theautonomous flight path. In FIG. 6, the forward velocity component mayrefer to a velocity component of the UAV along a direction parallel toautonomous flight path 608. In some instances, the forward velocitycomponent may refer to a velocity component of the UAV along the rollaxis of the UAV. In such cases, the velocity component that is added tothe UAV as a result of the actuation of the throttle stick may bedescribed by the equation: (2) added velocity component=(roll stickposition)×(forward velocity component)×factor. A negative velocity mayindicate that the UAV is moving down while a positive velocity componentmay indicate that the UAV is moving up. In some instances, the factormay be equal to or less than about 0.1, 0.2, 0.4, 0.6, 0.8, 1, 2, or 4.In some instances, the factor may be equal to about 0.5.

In some instances, components (e.g., velocity component, accelerationcomponent, etc) may be added without affecting a flight path of the UAV.FIG. 14 illustrates a top-down view of an unmanned aerial vehicle movingwith increased or decreased velocity along an autonomous flight path, inaccordance with embodiments. In some instances, a user input may modifya velocity or acceleration of an autonomously flying UAV withoutaffecting its trajectory or flight path. In some instances, the userinput to affect velocity of the UAV may be received via one or morecontrol sticks.

In some instances, the control stick configured to affect a velocity ofthe UAV without affecting a flight path of the UAV may be a pitch stick.For example, control stick 1402 may be an example of a pitch stick.Alternatively, control stick 1403 may be an example of a pitch stick.Actuation of the pitch stick may affect or modify a velocity of the UAV,e.g., along a roll axis of the UAV. In some instances, the velocity ofthe UAV may be increased or decreased according to a degree of actuationof the pitch stick. For example, the pitch stick may comprise a restingstate where no force is exerted on the pitch stick, and two fullyactuated states in opposite directions. For example, the pitch stick maybe configured to move towards the bottom as indicated by direction 1404and towards the top. In some instances, the position of the pitch stickmay be described as being between −1 (fully actuated to the bottom) and1 (fully actuated to the top). For example, a pitch stick that ishalfway moved towards the bottom may comprise a position of −0.5 while aroll stick that is a third of the way moved towards the top may comprisea position of 0.333. In some instances, a velocity component of the UAV(e.g., along the autonomous flight path or along its roll direction)resulting from actuation of the pitch stick may be described by theequation: (1) velocity component=(pitch stick position)×velocityfactor+(basic flight velocity) as illustrated in embodiment 1406. Insome instances, the velocity factor may be a predetermined velocityvalue, e.g., factory set or user determined value. For example, thepredetermined velocity value may be equal to or less than about 2 m/s, 4m/s, 6 m/s, 8 m/s, 10 m/s, 12 m/s, 15 m/s, 20 m/s, or 50 m/s. In someinstances, the basic flight velocity may be a predetermined basic flightvelocity, e.g., factory set or user determined value. For example,predetermined basic flight velocity the predetermined velocity value maybe equal to or less than about 2 m/s, 4 m/s, 6 m/s, 8 m/s, 10 m/s, 12m/s, 15 m/s, 20 m/s, or 50 m/s. In some instances, a velocity componentof the UAV (e.g., along the autonomous flight path or along its rolldirection) resulting from actuation of the pitch stick may be describedby the equation: (2) velocity component=(pitch stick position+1)×(basicflight velocity)×factor as illustrated in embodiment. 1408. In someinstances, the factor may be equal to or less than about 0.1, 0.2, 0.4,0.6, 0.8, 1, 2, or 4. In some instances, the factor may be equal toabout 1.

In some instances, different control sticks may modify the flight pathof the movable object in different ways. For example, actuation of theroll stick may affect a rotation of the UAV about a roll axis which mayadd a horizontal velocity component to the UAV to modify the autonomousflight path of the UAV while actuation of the yaw stick may affectrotation of the UAV about a yaw axis which may add an acceleration(e.g., centripetal acceleration) component to the UAV to modify theautonomous flight path of the UAV. For example, actuation of thethrottle stick may add a vertical velocity component to the UAV tomodify the autonomous flight path of the UAV while actuation of thepitch stick may affect a velocity of the UAV about axis.

In some instances, a degree of the user input may correspond with howmuch the autonomous flight of the UAV is modified. The degree of userinput may correspond to a duration of the user input. For example, inresponse to continued input from a user continued actuation of the oneor more control sticks), the rotation of the UAV about one or more ofits axes may increase. For example, depending on an amount of time theone or more control sticks is actuated, a directional, velocity, and/oracceleration component that is added may continue to increase. In someinstances, the degree of user input may correspond to a force of theuser input. FIG. 7 illustrates a force of the user input proportionallymodifying the autonomous flight of the movable object, in accordancewith embodiments. In some instances, the degree of user input maycorrespond to an amount of force exerted by the user in providing theuser input. In some instances, the degree of user input may correspondto how much an input device (e.g., the one or more control sticks) isactuated. During a first time period 701, the degree of the user inputmay be at a first level 703. Accordingly, the autonomous flight path 704of the UAV 706 may be modified by a first degree as shown by themodified flight path 705. During a second time period 707, the degree ofuser input may be at a second level 709. For example, the user may exertmore force on a control stick or may actuate the control stick further.Accordingly, the autonomous flight path of the UAV may be modified by asecond degree as shown by the second modified flight path 711. In someinstances, the correspondence may be linear. For example, if the userexerts twice the amount of force, or actuates the control stick twice asfar, velocity components that are added to the UAV may be twice ascompared to before, and the flight path of the UAV may be modified bytwice the amount. Alternatively, the correspondence may not be linear.In some instances, the relationship between the actuation of the one ormore control sticks and behavior of the UAV may be as previouslydescribed herein, e.g., with respect to the roll stick, pitch stick, andyaw stick.

In some instances, the autonomous flight of the UAV may be modifiedtaking into account various other factors addition to the user input).In some instances, the various other factors may include environmentalfactors. The environmental factors may be determined based on one ormore sensors on board the UAV. For example, the autonomous flight pathof the UAV may be modified taking into account data from proximity orobstacle sensors in order to ensure that the desired modificationaccording to the user input will not endanger the UAV or others (e.g.,by putting the UAV at risk of colliding with a detected obstacle). Forexample, the autonomous flight path of the UAV may be modified takinginto account data from other sensors in order to ensure that the desiredmodification according to the user input will not destabilize a balanceof the UAV such that UAV becomes unstable or uncontrollable.Accordingly, the user input may be processed and interpreted by a flightcontroller on board the UAV before signals are generated andinstructions are sent to the one or more propulsion mechanism to modifythe autonomous flight path, e.g., to ensure stability and safety.

In some instances, the various factors may referred to above may includea threshold. The threshold may be a threshold time and/or a thresholddistance. The threshold time and/or threshold distance may bepredetermined. In some instances, the threshold time and/or thresholddistance may be configured prior to and/or during operation of the UAVby a user. FIG. 8 illustrates behaviors of the UAV upon reaching athreshold, in accordance with embodiments. In some instances, thethreshold may be a threshold distance. For example, if the UAV's flightpath deviates from the original autonomous flight path by more than athreshold distance as a result of a user input, further modification ofthe autonomous flight path may be prevented, despite the user input. Insome instances, the threshold may be a threshold time. For example, ifthe UAV's flight path deviates from the original autonomous flight pathfor more than a threshold time as a result of a user input, furtherdeviation may be prevented, despite the user input.

For example, autonomous flight of a UAV 812 may be effected. Theautonomous flight may comprise an autonomous flight path 814 towards atarget 816. While the UAV is autonomously flying towards the target, auser (e.g., operator of the UAV) may modify the autonomous flight, e.g.,by actuating one or more control sticks. The UAV may follow a modifiedflight path 818 as a result of the user input. In some instances, oncethe UAV is at a threshold distance 819 away from the original autonomousflight path, the user input may no longer instruct the UAV to deviatefrom the autonomous flight path 814. For example, even with continueduser input, the UAV may only be able maintain the threshold distance 819from the autonomous flight path 814 as shown in embodiment 810. Forexample, once the UAV reaches the threshold distance as a result of theuser input, the UAV may begin moving towards the target 826 as shown inembodiment 820. In some instances, once the UAV reaches the thresholddistance as a result of the user input, the UAV may begin moving towardsthe original autonomous flight path 824 before moving towards thetarget. In some instances, once the UAV reaches the threshold distanceas a result of the user input, the autonomous flight of the UAV may bedisrupted as shown in embodiment 830. The UAV may hover at the location832 afterwards, or land at or near the location where the distancethreshold was reached. In some instances, the user may be required tomanually control the UAV after reaching the threshold distance away fromthe autonomous flight path.

In some instances, an alert may be sent to the user upon reaching thethreshold distance. The alert may be a visual, auditory, and/or tactilealert. In some instances, the alert may be sent to the user terminal.The alert may inform the user that the user is at the thresholddistance. In some instances, the alert may inform the user thatcontinued user input and/or deviation from the original autonomousflight path will result in termination of the autonomous flight. In someinstances, the alert may be sent upon reaching the threshold distancebut the autonomous flight of the UAV may continue to be modified by theuser input. In some instances, the alert may be sent upon reaching thethreshold distance and further modification of the flight path may beprevented as previously described in embodiments 810, 820, or 830. Insome instances, the behavior of the UAV may depend on more than onethreshold. For example, the behavior of the UAV after release of theuser input may depend on both a threshold time and a threshold distance.For example, the behavior of the UAV during the user input may beaffected by a first threshold distance (or time) and a second thresholddistance (or time). In some instances, the UAV may send an alert to theuser in deviating from the autonomous flight path by a first thresholddistance and may be prevented from further deviation upon reaching asecond threshold distance greater than the first threshold distance.

While threshold distances have primarily been discussed herein, it shallbe understood that the discussions above may be equally applicable forthreshold times. For example, if the UAV deviates from the autonomousflight path for a duration longer than the threshold time, the UAV maybe forced to return to the original autonomous flight path before beingallowed to deviate once again. In some instances, if the UAV's flightpath deviates from the autonomous flight path for a duration longer thanthe threshold time, an alert may be sent to the user as described above.

The modification of the autonomous flight ay be maintained only for atime period during which the user input is maintained. Alternatively,the modification of the autonomous flight path may be maintained afterthe user input. FIG. 9 illustrates behavior of the UAV after user inputmodifying the autonomous flight of the movable object is released, inaccordance with embodiments. For each of the configurations 910, 920,930, 940, and 950, user input modifying the autonomous flight path maybe received for a time period T1. The user input may be released afterthe time period. After the release, the movable object may autonomouslyoperate to fly towards the target (e.g., target object and/or targetdestination) as shown by embodiments 910 and 920. In some instances, theUAV may calculate (e.g., autonomously calculate) the shortest flightpath between the UAV and the target and generate a new autonomous flightpath 912 to take in moving towards the target. In some instances, theUAV may return to the original autonomous flight path 922 and continueits autonomous flight towards the target. Alternatively, after therelease, the UAV may continue its flight on the modified flight path932. For example, after actuation of yaw stick and subsequent release,the UAV may fly along its new roll direction. In some instances, afterthe release, the UAV may remain at the location 942 where the user inputis released until further input. For example, the UAV may hover at thelocation or may land at, or near, the location where the user input isreleased. In some instances, after release, the UAV may take a newautonomous flight path 952. The new autonomous flight path may beparallel to the original autonomous flight path. Alternatively, the newautonomous flight path may be at any arbitrary angle with respect to theoriginal autonomous flight path.

In some instances, the behavior of the UAV after release of the userinput (e.g., that modifies the autonomous flight path) may depend on athreshold. The threshold may be a threshold time and/or a thresholddistance. For example, if the user input is provided for a duration morethan a threshold time, after release of the user input, the UAV maycontinue on the modified autonomous flight path as shown inconfiguration 930. However, if the user input is provided for a durationless than the threshold time, the UAV may autonomously operate towardsthe target as shown in configurations 910 or 920. In some instances, ifthe user input is provided for a duration less than the threshold time,the UAV may remain at the location where the user input is releaseduntil further instructions are given as shown in configuration 940. Thethreshold distance and/or threshold time may be predetermined. In someinstances, the threshold distance and/or threshold time may be userconfigured prior to and/or during operation of the UAV.

While threshold times have primarily been discussed, it shall beunderstood that the discussions above may be equally applicable forthreshold distances. For example, if the user input is provided suchthat the UAV deviates from the original autonomous flight path by morethan a distance threshold, the UAV may continue on the modifiedautonomous flight path as shown in configuration 930 after release ofthe user input. However, if the user input is provided such that the UAVdeviates from the original autonomous flight path less than a distancethreshold, the UAV may autonomously operate towards the target as shownin configurations 910 or 920 or may remain at the location where theuser input is released as shown in configuration 940.

Any combination of the various configurations provided herein may bepossible. For example, if the user input is provided for a duration lessthan a threshold time, the UAV may continue on the modified autonomousflight path as shown in configuration 930 while if the user input ismaintained for a duration more than the threshold time, the UAV mayautonomously operate towards the original target as shown inconfigurations 910 or 920. In addition, the behavior of the UAV maydepend on more than one threshold. For example, the behavior of the UAVafter release of the user input may depend on both a threshold time anda threshold distance. For example, the behavior of the UAV after releaseof the user input may depend on whether the user input is maintained fora duration more than a first threshold time (or distance) or a secondthreshold time (or distance).

Once the UAV begins following a new autonomous flight path, subsequentuser input during autonomous flight may modify the new autonomous flightpath. Alternatively, subsequent user input during autonomous flight maymodify the UAV with respect to the original autonomous flight path. FIG.10 illustrates a new autonomous flight path of a UAV modified by a userinput, in accordance with embodiments. Substantially as describedherein, autonomous flight of UAV 1002 may be effected. The autonomousflight of the UAV may comprise an original autonomous flight path 1004.During a time period T1, a user input may modify the autonomous flightpath of the UAV. After release of the user input, the UAV may continueits autonomous flight. In some instances, the UAV may continue itsautonomous flight with a new autonomous flight path 1006. During a timeperiod T2, a new user input may modify the new autonomous flight path ofthe UAV. In some instances, the modification 1008 may be with respect tothe new autonomous flight path, as shown in embodiment 1010. Forexample, a direction component may be added to the new autonomous flightpath. The direction component may be perpendicular to the new autonomousflight path. In some instances, the direction component may be addedalong a reference plane such as a transverse plane of the UAV or along ahorizontal plane. Alternatively or in addition, a velocity component maybe added to the UAV in modifying the new autonomous flight path of theUAV. The velocity component may be perpendicular to the new autonomousflight path of the UAV. In some instances, the velocity component may beadded along a reference plane such as a transverse plane of the UAV oralong a horizontal plane. Alternatively or in addition, an accelerationcomponent may be added to the UAV in modifying the new autonomous flightpath of the UAV. The acceleration component may be perpendicular to thenew autonomous flight path of the UAV. In some instances, theacceleration component may be added along a reference plane such as atransverse plane of the UAV or along a horizontal plane.

In some instances, the modification 1022 may be with respect to theoriginal autonomous flight path 1024 as shown in embodiment 1020. Forexample, a direction component may be added to the original autonomousflight path. The direction component may be perpendicular to theoriginal autonomous flight path. In some instances, the directioncomponent may be added along a reference plane such as a transverseplane of the UAV or along a horizontal plane. Alternatively or inaddition, a velocity component may be added to the UAV in modifying thenew autonomous flight path of the UAV. The velocity component may beperpendicular to the original autonomous flight path of the UAV. In someinstances, the velocity component may be added along a reference planesuch as a transverse plane of the UAV or along a horizontal plane.Alternatively or in addition, an acceleration component may be added tothe UAV in modifying the new autonomous flight path of the UAV. Theacceleration component may be perpendicular to the original autonomousflight path of the UAV. In some instances, the acceleration componentmay be added along a reference plane such as a transverse plane of theUAV or along a horizontal plane.

In some instances, the one or more control sticks described throughoutmay be used in conjunction to affect operation of the UAV. For example,the one or more control sticks may be used simultaneously or insequence. In some instances, the one or more control sticks may be usedtogether in order to adjust autonomous flight of the UAV. In someinstances, the yaw stick may be used together with the roll sticks, thepitch stick, and/or the throttle stick. In some instances, the rollstick may be used together with the pitch stick, the throttle stick,and/or the yaw stick. In some instances, the pitch stick may be usedtogether with the throttle stick, the yaw stick, and/or the roll stick.In some instances, the throttle stick may be used together with the yawstick, the roll stick, and/or the pitch stick. In some instances, twomeans of input (e.g., two control sticks) may be operated by a user toaffect or change behavior of the UAV during autonomous flight. In someinstances, 3, 4, 5, 6, 7, 8, 9, 10 or more means of input may beoperated by a user to affect or change behavior of the UAV duringautonomous flight of the UAV. The means of input may include, but arenot be limited to, control sticks, buttons, accelerometers, voice inputdevices, etc, substantially as described elsewhere. The combination ofuser input provided on the user terminal (e.g., on control sticks) whileautonomous flight of the UAV is effected may further modify autonomousflight of the UAV in novel ways.

For example, while the UAV is under autonomous flight and flying in acertain direction (e.g., under tap-to-go operation), a user of the UAVmay operate a control stick (e.g., a yaw stick) that affects rotation ofthe UAV around the yaw axis and a control stick (e.g., a throttle stick)that affects a height of the UAV. In such a case, the UAV may circularlyascend or descend. If the user's inputs are maintained on each of thecontrol sticks, the UAV may spirally ascend or descend. For example,while the UAV is under autonomous flight (e.g., under tap-to-gooperation), a user of the UAV may operate a control stick (e.g., a yawstick) that affects rotation of the UAV around the yaw axis and acontrol stick (e.g., a pitch stick) that affects a velocity of the UAV.In such a case, a turn of the UAV may be precisely controlled. Forexample, while under autonomous flight, the yaw stick may be fullyactuated (to the left) while the pitch stick is fully actuated to thebottom to decrease velocity of the UAV. Radius of the curvature may befurther decreased than if the yaw stick was actuated.

The systems provided herein may enable users to rapidly change atrajectory of the UAV while maintaining overall autonomous control(e.g., by a flight controller) such that the UAV is able to continue incompleting its objective or continue flying towards a target. Seamlesstransition between autonomous flight and modification of the autonomousflight may be possible such that the burden of manually piloting theaerial vehicle on the user can be significantly reduced, while stillenabling a degree of control by the user when desired or advantageous.Alternatively or in addition, the different user interfaces may enableusers to quickly and intuitively react to react to emergencies orunanticipated situations (e.g., by disrupting autonomous flight ormodifying the autonomous flight).

The systems, devices, and methods described herein can be applied to awide variety of movable objects. As previously mentioned, anydescription herein of an aerial vehicle may apply to and be used for anymovable object. A movable object of the present disclosure can beconfigured to move within any suitable environment, such as in air(e.g., a fixed-wing aircraft, a rotary-wing aircraft, or an aircrafthaving neither fixed wings nor rotary wings), in water (e.g., a ship ora submarine), on ground (e a motor vehicle, such as a car, truck, bus,van, motorcycle; a movable structure or frame such as a stick, fishingpole; or a train), under the ground (e.g., a subway), in space (e.g., aspaceplane, a satellite, or a probe), or any combination of theseenvironments. The movable object can be a vehicle, such as a vehicledescribed elsewhere herein. In some embodiments, the movable object canbe mounted on a living subject, such as a human or an animal. Suitableanimals can include avines, canines, felines, equines, bovines, ovines,porcines, delphines, rodents, or insects.

The movable object may be capable of moving freely within theenvironment with respect to six degrees of freedom (e.g., three degreesof freedom in translation and three degrees of freedom in rotation).Alternatively, the movement of the movable object can be constrainedwith respect to one or more degrees of freedom, such as by apredetermined path, track, or orientation. The movement can actuated byany suitable actuation mechanism, such as an engine or a motor. Theactuation mechanism of the movable object can be powered by any suitableenergy source, such as electrical energy, magnetic energy, solar energy,wind energy, gravitational energy, chemical energy, nuclear energy, orany suitable combination thereof. The movable object may beself-propelled via a propulsion system, as described elsewhere herein.The propulsion system may optionally run on an energy source, such aselectrical energy, magnetic energy, solar energy, wind energy,gravitational energy, chemical energy, nuclear energy, or any suitablecombination thereof. Alternatively, the movable object may be carried bya living being.

In some instances, the movable object can be a vehicle. Suitablevehicles may include water vehicles, aerial vehicles, space vehicles, orground vehicles. For example, aerial vehicles may be fixed-wing aircraft(e.g., airplane, gliders), rotary-wing aircraft (e.g., helicopters,rotorcraft), aircraft having both fixed wings and rotary wings, oraircraft having neither (e.g., blimps, hot air balloons). A vehicle canbe self-propelled, such as self-propelled through the air, on or inwater, in space, or on or under the ground. A self-propelled vehicle canutilize a propulsion system, such as a propulsion system including oneor more engines, motors, wheels, axles, magnets, rotors, propellers,blades, nozzles, or any suitable combination thereof. In some instances,the propulsion system can be used to enable the movable object to takeoff from a surface, land on a surface, maintain its current positionand/or orientation (e.g., hover), change orientation, and/or changeposition.

The movable object can be controlled remotely by a user or controlledlocally by an occupant within or on the movable object. In someembodiments, the movable object is an unmanned movable object, such as aUAV. An unmanned movable object, such as a UAV, may not have an occupantonboard the movable object. The movable object can be controlled by ahuman or an autonomous control system (e.g., a computer control system),or any suitable combination thereof. The movable object can be anautonomous or semi-autonomous robot, such as a robot configured with anartificial intelligence.

The movable object can have any suitable size and/or dimensions. In someembodiments, the movable object may be of a size and/or dimensions tohave a human occupant within or on the vehicle. Alternatively, themovable object may be of size and/or dimensions smaller than thatcapable of having a human occupant within or on the vehicle. The movableobject may be of a size and/or dimensions suitable for being lifted orcarried by a human. Alternatively, the movable object may be larger thana size and/or dimensions suitable for being lifted or carried by ahuman. In some instances, the movable object may have a maximumdimension (e.g., length, width, height, diameter, diagonal) of less thanor equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 5 m, or 10 m. Themaximum dimension may be greater than or equal to about: 2 cm, 5 cm, 10cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. For example, the distance betweenshafts of opposite rotors of the movable object may be less than orequal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m.Alternatively, the distance between shafts of opposite rotors may begreater than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, or 10 m.

In some embodiments, the movable object may have a volume of less than100 cm×100 cm×100 cm, less than 50 cm×50 cm×30 cm, or less than 5 cm×5cm×3 cm. The total volume of the movable object may be less than orequal to about: 1 cm³, 2 cm³, 5 cm³, 10 cm³, 20 cm³, 30 cm³, 40 cm³, 50cm³, 60 cm³, 70 cm³, 80 cm³, 90 cm³, 100 cm³, 150 cm³, 200 cm³, 300 cm³,500 cm³, 750 cm³, 1000 cm³, 5000 cm³, 10,000 cm³, 100,000 cm³, 1 m³, or10 m³. Conversely, the total volume of the movable object may be greaterthan or equal to about: 1 cm³, 2 cm³, 5 cm³, 10 cm³, 20 cm³, 30 cm³, 40cm³, 50 cm³, 60 cm³, 70 cm³, 80 cm³, 90 cm³, 100 cm³, 150 cm³, 200 cm³,300 cm³, 500 cm³, 750 cm³, 1000 cm³, 5000 cm³, 10,000 cm³, 100,000 cm³,1 m³, or 10 m³.

In some embodiments, the movable object may have a footprint (which mayrefer to the lateral cross-sectional area encompassed by the movableobjectless than or equal to about: 32,000 cm², 20,000 cm², 10,000 cm²,1,000 cm², 500 cm², 100 cm², 50 cm², 10 cm², or 5 cm². Conversely, thefootprint may be greater than or equal to about: 32,000 cm², 20,000 cm²,10,000 cm², 1,000 cm², 500 cm², 100 cm², 50 cm², 10 cm², or 5 cm².

In some instances, the movable object may weigh no more than 1000 kg.The weight of the movable object may be less than or equal to about:1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60 kg, 50kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10 kg, 9 kg,8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1 kg, 0.05 kg,or 0.01 kg. Conversely, the weight may be greater than or equal toabout: 1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60kg, 50 kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10kg, 9 kg, 8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1kg, 0.05 kg, or 0.01 kg.

In some embodiments, a movable object may be small relative to a loadcarried by the movable object. The load may include a payload and/or acarrier, as described in further detail below. In some examples, a ratioof a movable object weight to a load weight may be greater than, lessthan, or equal to about 1:1. In some instances, a ratio of a movableobject weight to a load weight may be greater than, less than, or equalto about 1:1. Optionally, a ratio of a carrier weight to a load weightmay be greater than, less than, or equal to about 1:1. When desired, theratio of an movable object weight to a load weight may be less than orequal to: 1:2, 1:3, 1:4, 1:5, 1:10, or even less. Conversely, the ratioof a movable object weight to a load weight can also be greater than orequal to: 2:1, 3:1, 4:1, 5:1, 10:1, or even greater.

In some embodiments, the movable object may have low energy consumption.For example, the movable object may use less than about: 5 W/h, 4 W/h, 3W/h, 2 W/h, 1 W/h, or less. In some instances, a carrier of the movableobject may have low energy consumption. For example, the carrier may useless than about: 5 W/h, 4 W/h, 3 W/h, 2 W/h, 1 W/h, or less. Optionally,a payload of the movable object may have low energy consumption, such asless than about: 5 W/h, 4 W/h, 3 W/h, 2 W/h, 1 W/h, or less.

FIG. 11 illustrates an unmanned aerial vehicle (UAV) 1100, in accordancewith embodiments. The UAV may be an example of a movable object asdescribed herein, to which the method and apparatus of discharging abattery assembly may be applied. The UAV 1100 can include a propulsionsystem having four rotors 1102, 1104, 1106, and 1108. Any number ofrotors may be provided (e.g., one, two, three, four, five, six, ormore). The rotors, rotor assemblies, or other propulsion systems of theunmanned aerial vehicle may enable the unmanned aerial vehicle tohover/maintain position, change orientation, and/or change location. Thedistance between shafts of opposite rotors can be any suitable length1110. For example, the length 1110 can be less than or equal to 2 m, orless than equal to 5 m. In some embodiments, the length 1110 can bewithin a range from 40 cm to 1 m, from 10 cm to 2 m, or from 5 cm to 5m. Any description herein of a UAV may apply to a movable object, suchas a movable object of a different type, and vice versa. The UAV may usean assisted takeoff system or method as described herein.

FIG. 12 is a schematic illustration by way of block diagram of a system1200 for controlling a movable object, in accordance with embodiments.The system 1200 can be used in combination with any suitable embodimentof the systems, devices, and methods disclosed herein. The system 1200can include a sensing module 1202, processing unit 1204, non-transitorycomputer readable medium 1206, control module 1208, and communicationmodule 1210.

The sensing module 1202 can utilize different types of sensors thatcollect information relating to the movable objects in different ways.Different types of sensors may sense different types of signals orsignals from different sources. For example, the sensors can includeinertial sensors, GPS sensors, proximity sensors (e.g., lidar), orvision/image sensors (e.g., a camera). The sensing module 1202 can beoperatively coupled to a processing unit 1204 having a plurality ofprocessors. In some embodiments, the sensing module can be operativelycoupled to a transmission module 1212 (e.g., a Wi-Fi image transmissionmodule) configured to directly transmit sensing data to a suitableexternal device or system. For example, the transmission module 1212 canbe used to transmit images captured by a camera of the sensing module1202 to a remote terminal.

The processing unit 1204 can have one or more processors, such as aprogrammable processor (e.g., a central processing unit (CPU)). Theprocessing unit 1204 can be operatively coupled to a non-transitorycomputer readable medium 1206. The non-transitory computer readablemedium 1206 can store logic, code, and/or program instructionsexecutable by the processing unit 1204 for performing one or more steps.The non-transitory computer readable medium can include one or morememory units (e.g., removable media or external storage such as an SDcard or random access memory (RAM)). In some embodiments, data from thesensing module 1202 can be directly conveyed to and stored within thememory units of the non-transitory computer readable medium 1206. Thememory units of the non-transitory computer readable medium 1206 canstore logic, code and/or program instructions executable by theprocessing unit 1204 to perform any suitable embodiment of the methodsdescribed herein. For example, the processing unit 1204 can beconfigured to execute instructions causing one or more processors of theprocessing unit 1204 to analyze sensing data produced by the sensingmodule. The memory units can store sensing data from the sensing moduleto be processed by the processing unit 1204. In some embodiments, thememory units of the non-transitory computer readable medium 1206 can beused to store the processing results produced by the processing unit1204.

In some embodiments, the processing unit 1204 can be operatively coupledto a control module 1208 configured to control a state of the movableobject. For example, the control module 1208 can be configured tocontrol the propulsion mechanisms of the movable object to adjust thespatial disposition, velocity, and/or acceleration of the movable objectwith respect to six degrees of freedom. Alternatively or in combination,the control module 1208 can control one or more of a state of a carrier,payload, or sensing module.

The processing unit 1204 can be operatively coupled to a communicationmodule 1210 configured to transmit and/or receive data from one or moreexternal devices (e.g., a terminal, display device, or other remotecontroller). Any suitable means of communication can be used, such aswired communication or wireless communication. For example, thecommunication module 1210 can utilize one or more of local area networks(LAN), wide area networks (WAN), infrared, radio, point-to-point (P2P)networks, telecommunication networks, cloud communication, and the like.Optionally, relay stations, such as towers, satellites, or mobilestations, can be used. Wireless communications can be proximitydependent or proximity independent. In some embodiments, line-of-sightmay or may not be required for communications. The communication module1210 can transmit and/or receive one or more of sensing data from thesensing module 1202, processing results produced by the processing unit1204, predetermined control data, user commands from a terminal orremote controller, and the like.

The components of the system 1200 can be arranged in any suitableconfiguration. For example, one or more of the components of the system1200 can be located on the movable object, carrier, payload, terminal,sensing system, or an additional external device in communication withone or more of the above. Additionally, although FIG. 12 depicts asingle processing unit 1204 and a single non-transitory computerreadable medium 1206, one of skill in the art would appreciate that thisis not intended to be limiting, and that the system 1200 can include aplurality of processing units and/or non-transitory computer readablemedia. In some embodiments, one or more of the plurality of processingunits and/or non-transitory computer readable media can be situated atdifferent locations, such as on the movable object, carrier, payload,terminal, sensing module, additional external device in communicationwith one or more of the above, or suitable combinations thereof, suchthat any suitable aspect of the processing and/or memory functionsperformed by the system 1200 can occur at one or more of theaforementioned locations.

As used herein A and/or B encompasses one or more of A or B, andcombinations thereof such as A and B. It will be understood thatalthough the terms “first,” “second,” “third” etc. may be used herein todescribe various elements, components, regions and/or sections, theseelements, components, regions and/or sections should not be limited bythese terms. These err are merely used to distinguish one element,component, region or section from another element, component, region orsection. Thus, a first element, component, region or section discussedbelow could be termed a second element, component, region or sectionwithout departing from the teachings of the present disclosure.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” or “includes” and/or “including,” when used in thisspecification, specify the presence of stated features, regions,integers, steps, operations, elements and/or components, but do notpreclude the presence or addition of one or more other features,regions, integers, steps, operations, elements, components and/or groupsthereof.

Furthermore, relative terms, such as “lower” or “bottom” and “upper” or“top” may be used herein to describe one element's relationship to otherelements as illustrated in the figures. It will be understood thatrelative terms are intended to encompass different orientations of theelements in addition to the orientation depicted in the figures. Forexample, if the element in one of the figures is turned over, elementsdescribed as being on the “lower” side of other elements would then beoriented on the “upper” side of the other elements. The exemplary term“lower” can, therefore, encompass both an orientation of “lower” and“upper,” depending upon the particular orientation of the figure.Similarly, if the element in one of the figures were turned over,elements described as “below” or “beneath” other elements would then beoriented “above” the other elements. The exemplary terms “below” or“beneath” can, therefore, encompass both an orientation of above andbelow.

While some embodiments of the present disclosure have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the disclosure. It should beunderstood that various alternatives to the embodiments of thedisclosure described herein may be employed in practicing thedisclosure. Numerous different combinations of embodiments describedherein are possible, and such combinations are considered part of thepresent disclosure. In addition, all features discussed in connectionwith any one embodiment herein can be readily adapted for use in otherembodiments herein. It is intended that the following claims define thescope of the invention and that methods and structures within the scopeof these claims and their equivalents be covered thereby.

What is claimed is:
 1. A system for controlling an unmanned aerialvehicle (UAV), comprising: a first user interface configured to receivea first user input, the first user input providing one or moreinstructions to effect an autonomous flight of the UAV towards a target,the first user input comprising information regarding a first flightparameter and a second flight parameter controlling the autonomousflight of the UAV, and the first flight parameter being associated withat least one of a direction of a flight path of the UAV, a velocity ofthe UAV, or an acceleration of the UAV; and a second user interfaceconfigured to receive a second user input, the second user inputproviding one or more instructions to modify the first flight parameterto be a modified first flight parameter that is configured to alter theat least one of the direction of the flight path of the UAV, thevelocity of the UAV, or the acceleration of the UAV to modify theautonomous flight of the UAV, wherein: the second flight parameterincludes a flight parameter, associated with the first user input, thatis unchanged by the second user input; and at least during a duration ofthe second user input, a flight of the UAV is determined according tothe modified first flight parameter and the second flight parameter. 2.The system of claim 1, wherein the target includes a target object or atarget destination.
 3. The system of claim 1, wherein the autonomousflight includes at least one of: a flight to a predetermined location;an autonomous return of the UAV; an autonomous navigation along one ormore waypoints; an autonomous flight to a point of interest; a flightalong a preset trajectory or a preset direction; a flight along anautonomously planned trajectory; a flight along a user configuredtrajectory; a flight to a tapped location on a map of the first userinterface; or a flight tracking a target object.
 4. The system of claim1, wherein the first user interface is located on a first device and thesecond user interface is located on a second device.
 5. The system ofclaim 4, wherein: the first device or the second device includes ahandheld device or a mobile device, the handheld or mobile deviceincluding a cell phone, tablet, or PDA; and the second device includes aremote controller.
 6. The system of claim 4, wherein the first deviceand the second device are operably coupled to one another.
 7. The systemof claim 1, wherein the first user interface and the second userinterface are located on a single device.
 8. The system of claim 1,wherein: the first user interface includes a touch screen; and the firstuser input is received via a user tapping on the touch screen.
 9. Thesystem of claim 1, wherein the first user interface is configured toshow a location of the UAV on a two dimensional map and/or displayimages received from a camera coupled to the UAV.
 10. The system ofclaim 1, wherein the second user interface includes one or moreactuatable mechanisms, the one or more mechanisms including one or morecontrol sticks.
 11. The system of claim 10, wherein: the one or morecontrol sticks include at least one of a roll stick configured to affecta rotation of the UAV about a roll axis and a yaw stick configured toaffect a rotation of the UAV about a yaw axis.
 12. The system of claim11, wherein an actuation of the one or more control sticks is configuredto effect at least one of: adding a direction component along ahorizontal plane and perpendicular to an autonomous flight path of theUAV, a degree of the actuation corresponding to a magnitude of thedirection component; adding a velocity component along the horizontalplane and perpendicular to the autonomous flight path of the UAV, adegree of the actuation corresponding to a magnitude of the velocitycomponent; adding an acceleration component to the autonomous flightpath of the UAV, a degree of the actuation corresponding to a magnitudeof the acceleration component; or adding a centripetal acceleration tothe UAV, a degree of the actuation inversely corresponding to a size ofa radius of a trajectory arc of the UAV.
 13. The system of claim 11,wherein the one or more control sticks is configured to stop theautonomous flight.
 14. The system of claim 1, wherein the first userinterface is further configured to display at least one of an autonomousflight path of the UAV or a modified flight path of the UAV.
 15. Thesystem of claim 1, wherein the second user input is configured to modifythe autonomous flight of the UAV for the duration of the second userinput.
 16. The system of claim 1, wherein the autonomous flight of theUAV is modified further taking into account environmental factors inconcert with the second user input, the environmental factors beingdetermined based on one or more sensors on board the UAV, the one ormore sensors including a camera.
 17. The system of claim 1, wherein theautonomous flight of the UAV is modified by a flight controller thattakes into account the second user input.
 18. The system of claim 1,wherein after the second user input is received and released, the UAV isconfigured to: fly along a prior flight path it was flying along priorto receiving the one or more instructions to modify the autonomousflight of the UAV; or fly along a new flight path different from theprior flight path.
 19. A method of controlling an unmanned aerialvehicle (UAV), comprising: receiving a first user input at a first userinterface, the first user input providing one or more instructions toeffect an autonomous flight of the UAV towards a target, the first userinput comprising information regarding a first flight parameter and asecond flight parameter controlling the autonomous flight of the UAV,and the first flight parameter being associated with at least one of adirection of a flight path of the UAV, a velocity of the UAV, or anacceleration of the UAV; and receiving a second user input at a seconduser interface, the second user input providing one or more instructionsto modify the first flight parameter to be a modified first flightparameter that is configured to alter the at least one of the directionof the flight path of the UAV, the velocity of the UAV, or theacceleration of the UAV to modify the autonomous flight of the UAV,wherein: the second flight parameter includes a flight parameter,associated with the first user input, that is unchanged by the seconduser input; and at least during a duration of the second user input, aflight of the UAV is determined according to the modified first flightparameter and the second flight parameter.
 20. A non-transitory computerreadable medium storing code, logic, or instructions to: receive a firstuser input at a first user interface, the first user input providing oneor more instructions to effect an autonomous flight of an unmannedaerial vehicle (UAV) towards a target, the first user input comprisinginformation regarding a first flight parameter and a second flightparameter controlling the autonomous flight of the UAV, and the firstflight parameter being associated with at least one of a direction of aflight path of the UAV, a velocity of the UAV, or an acceleration of theUAV; and receive a second user input at a second user interface, thesecond user input providing one or more instructions to modify the firstflight parameter to be a modified first flight parameter that isconfigured to alter the at least one of the direction of the flight pathof the UAV, the velocity of the UAV, or the acceleration of the UAV tomodify the autonomous flight of the UAV, wherein: the second flightparameter includes a flight parameter, associated with the first userinput, that is unchanged by the second user input; and at least during aduration of the second user input, a flight of the UAV being determinedaccording to the modified first flight parameter and the second flightparameter that is unchanged.